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FOREWORD 



ru- * f I rair ) n ^ Anal y sis and Evaluation Group (TAEG) was tasked by the / 
Chief of Naval Education and Training (CNET) to. conduct a study to determine' 
the relative effectiveness and efficiency of individualized instruction fori 
different kinds of training tasks and ability levels of trainees. A series 
of reports is being prepared to document different, specific aspects of th., 
TAEG program This is the first report in that series. It deafs with over- 
all, i.nterschool analyses of. training effectiveness and efficiency measures 
Subsequent reports in the series will present: ' f 



/ 



results of intraschool analyses of training effectiveness and / 
efficiency measures ' ■> 

a critical review of literature relevant to issues involved in'/ 
assessing instructional method effects on students '/ 

/ / 

results of analytical efforts concerning officer courses and an 
enlisted course that uses both individualized/and conventional' 
instruction. / / 
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SUMMARY OF THE STUDY 

The Chief of Naval Education and Training (CNET) tasked the Training 
Analysis and Evaluation Group (TAE6) to assess the effectiveness and effi- 
?] enc ^ °f individualized instruction (II) relative to conventional instrucr 
tion (CI). To the extent that differences in training effectiveness and/or 
efficiency were found to exist, the further purpose of the study was to 
determine if these differences are related to student ability levels and/or 
to specific types of training tasks. 

Two measures each of training effectiveness and training efficiency 
were examined. Student end-of-course^grades were used as an internal 
criterion of training effectiveness. Fleet supervisor ratings of the - 
adequacy of training for identified school'graduates were used as an 
external criterion of effectiveness.. Training efficiency measures were 
reflected by student time to complete training and by an individual training 
cost measure derived from total instructional costs for a given course. 
Student ability levels were represented by Armed Forces Qualification Test 

SIhI hSTh le / S n c5^i n f d t y convertin 9 their Armed Services Vocational 
Aptitude Battery (ASVAB) test scores. The instructional content of the 
various courses examined during the study was equated to a common .base by 
classifying learning objectives of each course into one of five generic task 
categories: fact, procedure, category, rule, principle. 

Nineteen Navy "A" schools were examined. Ten of the courses were con- 
ducted using II formats (eight were self-paced courses, two featured 
computer managed instruction). The other nine courses were conducted under 
conventional, group-paced instruction. 

A correlational approach was used to conduct the study. .Record data 
were col ected. on over 5,000 graduates of the 19 enlisted technical schools. 
The results of all .statistical analyses performed on the data are presented 
succinctly in section IV of the report. Major findings and conclusions of ■ 
the study are summarized below. 

1. Individualized instruction (self-paced and computer managed) and 
conventional group-paced instruction were found to be equally effective ways 
of preparing sailors for operational fleet jobs. This conclusion is based 
on comparisons of fleet supervisor ratings on 1,229 graduates of CI with 
ratings on 1,186 graduates of individualized courses. 

2. Within the II category, self-paced and computer managed 
instruction 'were found to be equally effective training methods as reflected 
by fleet supervisor ratings. 

3. Individualized instruction,, as it was conducted during the time 
period examined, benefited higher ability students during training more than 
it did lower ability students. Relative to the-lower ability students, 
higher ability personnel mastered more course content (i.e., had higher end- 
of-course grades) and completed training in less time. 

4. No definitive evidence was obtained concerning which method of 
instruction most benefits lower ability personnel. 
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5. Conventional instruction did not benefit one ability level of students 
over another during training. End-of-qourse grades for conventionally trained 
students, were equivalent across all ability levels. 

6. Fleet supervisor ratings were not dependent on school performance. 

No significant relationships were found between supervisor ratings and graduate 
end-of-course grades or time-to-complete training. 

7. Fleet supervisor ratings did not depend on ability level or sex of 
the graduate. % 

8. When course content is classified into generic training tasks, II 
was found to be more effective than CI in courses that taught primarily proce- 
dure tasks. Conventional instruction was more effective than II in courses 
that taught primarily rule or principle tasks. 

■a s. 

9. No one method of instruction was found to be universally more effec- 
tive in training all of the different types of tasks to different ability 
level 'students. The evidence of this study suggests that a combination of 
methods used within a given course for conveying different instructional 
contents would likely be more effective than use of a single method for an 
entire course. 

10. Definitive conclusions concerning the efficiency of II relative to 
CI could not be reached from the data of this study. 

The available data showed that overall training costs were higher for 
the II courses studied than for the CI courses. However, it cannot be con- 
cluded that II is more costly than CI since different courses with different 
contents and lengths were taught under each method. Similarly, time to com- 
plete training was found to be less overall for graduates of II courses than 
for those of CI courses. Again, however, because of differences in the courses 
studied; it could not be concluded from this study that individualizing instruc- 
tion necessarily reduces time required for training. 

, .Evidence was obtained, however, that bears on the time reduction. issue. 
Mental Category*! and 2 personnel required significantly less time to complete 
training under II than Mental Category 3 and 4 personnel (under II). From 
this, it can be inferred that converting current CI courses to an II format 
would likely reduce the time required for training format least the higher 
mental category groups (although the average course completion time cannot 
be specified). This inference is consonant with the findings of previous 
research which assert generally that student time is reduced when courses 
are converted from a CI to an II format— although the previous research did 
not relate the reductions to particular ability groups. 

General recommendations are made within the report concerning the use 
of II in Navy technical training. In addition, areas are identified where 
additional information is required to. develop a more complete understanding 
of factors involved in assessing and improving the. efficiency arid effective- 
ness of II. In this regard, two additional work efforts have been initiated 
by the TAEG. The first is concerned with identifying definitive baseline * 
estimates of average student time to complete training under II. The second 
.effort is concerned with improving the conduct of individualized instruction 
within the NAVEDTRACOM. 
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SECTION I 
INTRODUCTION 

BACKGROUND 

*w W A Tr? . review P f the status of individualized instruction fill with) 

ati SfeffSS ess^nS™^^ the 5- ee ' d / 0r an ^ "to Set" Sn r " 
ferent kind, nf W • • 9 "? e , fflclenc y ° f individualized instruction for dif- \ 

PURPOSE OF THIS STUDY 

efficient?? SuS P resent . stu ^ to assess the effectiveness and 
tha difference ?n tr in^° "J™- 10 "* 1 instruction (CI). To the extent 
oSJe* I S 2 n teaming effectiveness and/or efficiency were found to 

e S ilj eT e" S^rabllSJT 1 "? ff J 652 -"^fSre^d ?f r°- 
training tasks y 1eVe1S and/or t0 dif ^rent types of 

STUDY REPORTS 

A number of 'reports dealing with different, specific asDects of thp \ 
Pr ° 9ram are P lanned for publication. These 111 HSStlSS be?ow \ ' 



\ 
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. Two of the courses studied during the work program were taught using 
selfTpaced (SP) instruction at one location and group-paced (GP) instruction 
at another. One of these courses, Hull Maintenance Technician/ (HT "A"), is 
also dual-phased. The other, Damage Control Assistant (DCA), lis an officer 
course. These special courses required statistical treatments different 
from those used in other analyses. Consequently, the results /concerning 
instructional method effects on training effectiveness/efficiency obtained 
with these courses will be reported separately from the othen, investigations. 

f 

Another planned report will deal with the procedures used to equate the 
content of the different technical courses to a common base;/ i.e., into generic 
task classifications. It will also present information concerning differences 
in classifications made by school subject matter experts (SME) and research 
personnel. The results of comparisons between different training task classi- 
fication systems wi.ll also be presented. \ 1 

Finally, an executive summary of the entire study program will be prepared" 
It will summarize the major findings of the study and provide overall study . 
conclusions and recommendations. f 

ORGANIZATION OF THIS REPORT / 

The remainder of this report is presented in five sections and seven 
appendices. SectionUl presents the technical approach o ( 'f the study. Section 
III contains the results concerning the evidence for differential effective- 
ness and efficiency as related to the major variables examined: method of 
instruction, type training task, and student ability level. A discussion of 
the results and the interpretations that can be made are' given in section 
IV. Conclusions and recommendations are presented in section V. Section VI 
describes, additional work that has been initiated by the TAEG in the II area. 
/••.'. '" .. ' " /..'-.. 

Appendix A provides a concise. summary of previous /research findings 
concerning II versus, CI used in military settings. Information concerning 
"the Armed Services Vocational Aptitude Battery (ASVAB)/ is contained in appendix 
B. ASVAB scores were used as measures of student abil/ity for the present 
study. Appendix C presents the training package giver? to SMEs for classifying 
instructional content into generic task categories. Elements of the cost 
data base used in. assessing training efficiency are driscussed in appendix D. 
Appendix E identifies tfje schools studied during the /work program. Appendix 
F presents a description, of the hierarchical regression model used for data- 
analysis. Information obtained from analysis of missing data in the study 
is provided in appendix G. / 

\ 

/ 

/ 

■ -1 - / 
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SECTION II 

TECHNICAL APPROACH 

This section presents the technical approach to the study* A 
perspective on the study is presented .first. This is followed by a brief 
description of previous research concerning the effectiveness and efficiency 
of IT versus CI. Subsequently,- discussions of the variables examined, the 
samples of courses and graduates Jnyolved. in the present study, and the 
analytical design of the study are~provided. 

* *" 

STUDY PERSPECTIVE 

The study was designed to answer two major questions: 

1. Are significant, relative differences in training effectiveness 
and/or efficiency associated with individualized instruction versus .conven- 
tional instruction? , . J l. • 

2. If there are differences, are they related to different kinds of 
training tasks and/or ability l levels of trainees? 

Since experimental methods were impractical to employ within the 
context of an ongoing mil itary^ training system,* a correlational approach was 
used to" conduct the study! Under this approach, statistical analyses are 
performed on record data to determine if significant relationships exist 
among variables of interest. % . 

Within' the approach selected, the procedural steps consisted of r 

• / acquiring background information and knowledge 

• selecting relevant variables jncluding identifying sources of data 
on the variables ^ 

• selecting courses taught under each of the basic instructional " 
methods K ~ 

, selecting samples of graduates of the courses +. 

• .collecting appropriate data I j 

• analyzing the data. I r v: 
Each of these topics is discussed further below. " ^ 
BACKGROUND" x \r' 

At the beginning .of the program, "a brief literature review was 
conducted to obtain background information for the study. The review 
focused on a number of topics and issues relevant to the present^ 
investigation including: . / 

• findings of, and methodblbgies employed in, previous studies- 
conducted in both* academic and military environments concerning, 

- , the effects on training effectiveness/efficiency of different- 
instructional methods . 



various definitions and meanings of .the concept of ability and 
student achievements related to differing ability levels 
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• types of, and previous uses of, learning task classification 
. . systems 

• student learning of different* types of instructional content. 

Information obtained from the literature review was used in designing 
aspects of the present study, in developing study lines of inquiry, and in 
selecting the analytical strategy used. In addition to these uses, the body 
of knowledge available in the literature was fourjd to be of^ considerable 
general interest. Consequently, a separate technical ceport is being 
prepared to integrate and consolidate the i nf ormat iprf^ :ontained in the. 
^various sources. f * 

* Of jmrnedjate and direct interest to the present TAEG program, however, 
were those studies which compared the use of II versus CI in military 
settings. Orlansky and String" (1979) reviewed these efforts., Their review 
indicated that: " . \ 

X 

/ 

• II is as effective as CI in terms of student end-of -course 
achievement" scores 
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• the efficiency of II is significantly better than that of CI in 
terms of student time to complete instruction. 

• training costs for II are lower due largely to student time 
savings 

• the- addition of computer support to self -paced instruction does 
not significantly increase student time savings. 

Additional findings of the Orlansky and String review are presented in 
appendix A of this report. 

Orlansky and String noted that the previous studies. contrasting II jand 
CI were more or less controlled experimental studies, were of short 
duration, and used small numbers of subjects. The authors cited the need 
for information about the effects of II used over a longer period of time 
within an operational training context where situational factors differ 
considerably from those involved in the previous. research. The CNET tasking 
of TAEG required* an assessment of II effects within Naval Education and 
Training Command (NAVEDTRACOM) schools where both jnethods are used routinely 
for diverse skill and knowledge training. * Thus, the present TAEG study 

( complements previous research by assessing the general izability of prior 

1 findings. 

STUDY VARIABLES 

Seven major variables, three predictor and four criterion variables, 2 
Were selected for study. The variables, the measures of the variables 

2prect^e definitipns of these terms may be found in various statistical 

sources; ton^example, Finn (1974) and Cohen & Cohen (1975). 

(\ * . 
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employed, and the sources of data on the variables are identified in table 

1. Further discussions of these variables and\data sources are provide^; 

below- • j % J ; r~ 

PREDICTOR VARIABLES. The major predictor variables were method of instruction, 
trainee ability level, and type training task. \ j \ ] 

Method of Instruction . The primary predictor, variable of the study was instruc- 
tional method. Two basic methods were of interest : individualized instruction 
and conventional 1 instruction. Individualized; instruction involved seflf- *j 
paced (frequently referred to as IMI forj instructor -Managed instruction) and 
computer-managed courses (CMI). Conventional ; instruction involved group- | 
paced (GP) classroom instruction. \ / ~ % 7 5 
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Ability Level . Ability levels of trainees constituted the second/fnajor 
predictor variable. For the present study, ability levels of trainees were 
represented by student scores on the ASVAB. This test is\routyiely adminis- 
tered to all armed services enlistees. \ / ' 

The ASVAB consists of 12 subtests. The composition of ^e battery prior 
to October 1980 is shown in appendix B. Various combinations, or composites, 
of ASVAB subtest scores are used by the Navy to determine an individual's 
eligibility for attendance at specific technical schools. Scores from three 
of the battery subtests can also be used to derive an Armed Forces Qualifica- 
tion Test (AFQT) percentile score (see appendix B). These percentile scores 
provide a measure of general ability. They can also be used to group indivi- 
duals into mental categories. AFQT percentile scores (derived from the pre- 
October 1980 conversion routines) were used to represent ability levels of 
graduates for making comparisons across courses. A subsequent TAEG report 
will present relationships between composite scores and the other variables 
of interest. - m 

Types of Training Tasks . The third predictor variable was the types of tasks 
trained in school. Definitive conclusions concerning the value of II versus 
CI for training different types of tasks require that the same tasks be taught 
under each method. Unfortunately, the, same courses (and, consequently, the 
same tasks) are not taught under each, of the two basic methods of instruction, 
and it was beyond the scope of the study to create this condition experimentally, 
Thus, the matching of tasks under different methods of instruction was approxi- % 
mated by using a generic task classification system. The content of the 
different courses (i.e.. tasks taught) was equated to a common base so that 
appropriate comparisons could be made. The use of these comparisons assumes 
that the psychological processes involved in the acquisition of generic tasks 
•are similar regardless' of the specific context in which the learning occurs. 

} - 

For the present TAEG study, a modification of the Instructional Quality 
'inventory (IQI) method (Ellis, Wulfeck, & Fredericks, 1979) was used to equate 
skill and knowledge items taught in each of the courses studied to a common-; 
base. Subject matter experts at each school classified statements of the 
school's learning objectives into one of five types of information content: 
fact, category (.procedure, rule, and principle. The classifications are 
described in table 2.\ (See appendix C for a description of the*training 

\ 



Technical Report 117 
TABLE 1. MAJOR PREDICTOR AND CRITERION VARIABLES 

t t 



Vdr laD 1 e 


Measures 


Source 


rKhDILIOK 






MD1 1 1 Zy LeVB 1 


AFQT, Composites 


CNET 015 


Method of Instruction 


SP, CMI, GP 


CNET, CNTECHTRA 


Type of Training Task . 


Five Generic Tasks 

/ „ 


IQI (SMEs) 


CRITERION : *• 


i • 


;*".. ' 


EFFECTIVENESS 


r - 

• I 




• 

End-of -Course Grades 


Final Grades / 

/ 


School RprnrHsf^P + GP^ 
CNTECHTRA (CMI) 


Training Adequacy 


j 

i 
i 




Rat* innc 
•vet l 1 1 ly o 

i 


l-o ocaie / 

f „ 


TAS (CNET) 


' /EFFICIENCY. 

f 


i 




1 

1 Cost of Course 

/ 


Average Cost to 


CNET Accounting System 




Produce One; Graduate 




i 


(Per Session) 




Time to Complete 


Contact Hours 


CNTECHTRA (CMI), 






School Records, 

SMEs, NITRAS (SP + GP) 
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- 

TABLE 


2. INSTR'.'CTIONAL QUALITY INVENTORY CONTt-NT TYPES 


Content Type is: 




Fact: 


If the student must recall or recognize names, parts, 
.locations, functions, dates, places, etc. Example: name 
the parts of the 


Category: 


If the student must remember characteristics of similar 
objects, events, or ideas, OR if the student must sort or 
classify objects, events, or ideas according to character- 
istics. Example: identify target types from sonar 
signal displays. 


3 rorpdurp • 


If the student must remember a sequence of steps which 
apply to a single situation, OR if the student must apply 
the steps to a single piece of equipment or a single 
situation. Example: Field strip an M-16 rifle. 


Rule: 


If the, student, must remember a sequence of steps and 
decisions which apply in a variety of situations, OR if 
the student must apply the sequence across a variety of 
situations or types of equipment.' Example: Use 
mathematical formulas such as Ohm's law. 


Principle: 


If the student must remember how or why things work the <• 
way they do, or cause-effect relationships, OR if the 
student must use his knowledge to explain how things work 
or predict effects from causes. Example: Based on knowl- 
edge of electronic theory, predict effect on the 
' circuit if . 



Source: Ellis, Wulfeck, & Fredericks (1979) 
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package given to the SMEs for classifying course learning objectives using 
the IQI system.) 

In addition to those made by school SMEs, classifications of the same 
school -trained skill and knowledge items were also made (at TAEG request) by 
Navy Personnel Research and Development Center (NPRDC) staff. The NPRDC 
staff who made the classifications routinely use the IQI system and were 
involved in its development. Also, the questionnaire items were classified 
using a second task classification , system, TECEP (see Braby, Henry, Parrish, 
& Swope, 1978), to assess differences between classification schemes. Com- 
parative analyses of differences between raters using the same classification 
system and differences befween classification systems are- currently being 
performed. The results of these analyses will be provided in a later TAEG 
report. \ 

Miscellaneous Predictor Variables . In addition to the three primary predictor 
variables discussed above, the effects of two other variables, sex of the 
graduate and geographic location of the school(s), were also assessed. 

CRITERION VARIABLES. Two measures each of training e activeness and training 
efficiency were used as criterion variables. The effectiveness measures 
were end-of -course grades and fleet supervisor -ratings of school training 
adequacy. The efficiency measures were time-to-complete training and training 
costs. 

Training Effectiveness Measures . End-of-course grades which reflected how 
well' graduates performed in school were used as an internal measure of 
draining effectiveness. JHeet supervisor ratings which reflected graduate 
job performance were used as an external measure of course effectiveness. 
Internal measures of student learning achievement were available at the 1 
schools. External measures of student learning achievement were available 
at CNET. 

The external measures consisted of fleet supervisor ratings of the 
adequacy of school training for particular tasks which graduates are 
expected to perform on the job. External criteria of training effectiveness 
were of special interest to the study. The Orlansky and String (1979) 
review of II versus Cl^cbntrasts in military settings noted that'previous 
studies were confined^' to "comparisons on measures of student learning 
achievement available at the schools (see appendix A). External criteria of 
training effectiveness were apparently not/used (or were unavailable) for 
assessing the effects of different instrwctional methods. 

Fleet supervisors 1 ratings of training adequacy on identified courses and 
graduates were available at CNET in the NAVEDTRACOM Training Appraisal 
System (TAS) data base. The CNET Special Assistant for Training Appraisal 
(CNET 015') routinely collects feedback data, -via mailout questionnaires, 
from first-level fleet supervisors of recent technical school graduates. 
Random samples of graduates are drawn from the total pool of course 
graduates during a given time frame. 

Table 3 shows the type of data collected on* school graduates that was of 
interest to the study. Fleet supervisors rate on a 5-point scale the 
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adequacy of school training for an identified course graduate. Training 
adequacy judgments are made for a number of specific tasks for which a given 
technical school provided training. The task statements listed on a 
feedback questionnaire are currently prepared by technical training staff 
for a given course. The statements are based on the learning objectives of 
that course and thus reflect specific skills and knowledges taught at a 
school. 



TABLE 3. EXAMPLES OF TRAINING APPRAISAL SYSTEM DATA 



Adequacy of School Training: 

Circle One Number for Each Item Listed 










1. 

2. 
3. 
4. 
5. 


Unsatisfactory 
Less Than 1 Adequate 
Adequate - * 
More Than Adequate 
Much More Than Adequate 






• 




Skill or Knowledge Item 










(Fact) 


Identify Purposes and Organization of 
Personnel Qualification Standards 


1 2 


3 


4 


5 


(Category) 

r 

J 


Recognize Symptoms of Severe Electric 
Shock 


1 2 


3 


4 


5 


(Procedure) 


«* 

Operate Ship's Store Cash Register 


1 2,. 


3 


4 


5 


(Rule) 


Develop. Drawing Layout for Sheet 
Metal Projects 


1 2 


3 


4 


5 


(Principle) 


Troubleshoot Magnetic* Amplifiers 


1 2 


3 


4 


5 

■ 



The task statements shown in table 3 are typical of those used on TAS 
questionnaires. However, the particular statements in the table represent 
tasks of five different ratings. They were deliberately chosen to provide 
examples of the five generic task types. The classification names shown in 
parentheses in the table do not appear on TAS questionnaires. 

Listings of tasks taught at each school studied and supervisor ratings 
of training adequacy for these tasks were obtained from CNEJ. The question- 
naire task statements for each school were classified by appropriate school 
SMEs into the generic task categories' described earlier. This procedure 
permitted supervisor training adequacy ratings to be compiled for specific 
generic tasks. These data were then used for making "task-type" comparisons 
within and between courses. 
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Two different criterion measures were formed from the TAS fleet super- 
visor ratings for use in study analyses. One measure consisted of a mean 
TAS rating for each graduate involved in the study. Mean TAS ratings (for 
each graduate) were derived from the sum of all ratings assigned a given . 
graduate divided by the number of questionnaire items for which the graduate's 
supervisor assigned a rating. The second measure used was a task-specific 
TAS rating. Task -specific TAS ratings consisted of mean ratings for each 
generic task. The task-specific ratings were derived by summing the ratings 
pertaining to a given generic task and dividing by the number of ratings 
pertaining to that given type of task. * 

Training Efficiency Measures , Measures used to reflect training efficiency 
were training costs and student time (contact hours of instruction) to com- 
plete a course. ' % . • . 

Training cost data were available in the CNET Pfer Capita Cost Data Base 
for each course in the study. The training cost data were referenced to 
fiscal year 1979 which coincided with the time period that most graduates of 
interest to the study were 'in training. From the cost data base, a "training 
.cost_to_produce_oiie_gcaduate per course session" was derived arid used in 
subsequent analyses. Appendix D explains the derivation of training cost 
data. - * 

Student course completion times were available for individualized courses 
from CMI files and from class records at the schools for self -paced courses. 
Completion times for the conventional courses, were obtained from the schools , 
and from CNET computer files. ' _ 

STUDY SAMPLES 

. Two' sets of samples, were Required for the study. The first consisted 
of courses taught under each, of the basic methods of instruction, the second 
tif* graduates ,of these courses. ' 

COURSE SAMPLES. The TAEG project staff coordinated with CNET and Chief of 
Naval Technical. Trai ning (CNTECHTRA) staff to select courses. The plan was 
to identify approximately 10 A-level courses that used CI (i.e., group- 
paced) and 10 others that used II (i.e., self-paded or computer managed). 
Courses in each instructional category were to include the full range of 
ability levels of individuals who undergo Navy technical training." It was 
also desired that courses in each category be roughly matched on general 
instructional content (i.e., type training tasks) and on geographic location. 

i 

Initial selection of courses for inclusion in the method of instruction 
groups was made On the basis of entries in the Navy Integrated Training 
Resources and Administrative System (NITRAS). This system identifies courses 
that are considered to be individualized (SP or CMI) and those considered to 
be taught conventionally (GP). 

Courses selected for the study were primarily those classified as A-level ♦ 
A-level courses provide basic skill and knowledge training for entry level, 
Navy -jobs. These courses were Selected rather than C-level courses because: 
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a greater diversity of tasks is trained in the more general 
a- level courses than in the largely equipment-specific C-level 
courses * • 

a wider range of student abilities is involved in A-level training 
. - proportionately more A-level courses are taught under II. 

It was further desired that courses be selected to the extent DossiblP 
from those for which TAS data were already available or soon -to be aril able 
(i.e.,withm approximately the next 6 months). Although training appraisal 
f S S°" ld be altered, this would have resulted in lengthy timfdelayT 
ri^r^rlp^L 0 " -° U T grates. Consequently, availability of the TAS 
data became the major determinant of the sample composition. It is 
believed, however that the other criteria were reasonably well met and that 
the courses constituted a representative sample of Navy "A" schools The 

sJhSol Sw are Sh0Wn !J ^ e 4 ' The c ^ lete n«s of the courses and 
scnooi locations are provided in appendix E. 

s P 1f lTr n y~l h /t* A ;2? Vel COUrses were incllJ ded in the overall study. Nine- 
self -paced and two CMI courses were in the II group. Eleven qrouD-oaced 
courses were in the CI group. One Hull Techn1?1an P coursrSs 9 «Sdu??ed 
fo^ ?tndf "factional methods.- Two officer courses were also selected 
osten^M; tip" COnt K\ t ° f l hB COUrSeS ' Dama 9 e Contro1 Assistant, was 
insS on I T5rfS-n eaC 5 J° UrSe W \ S taught under a different method of 
2 2 °i t J" pm J 10 MA.. d ^ ta were obtained f ™ two basic CMI courses.- 
the RM Bas°cs r Jt M School and the RM Shore "A" School also attended 
tne km Basics course. Their records were obtained from CNTECHTRA CMI files 

m^oT^rn^^ *2 tf T Study ' Similarly ' graduates of hi 
(k) B a Src P ^' c ^ ? PS1 SCh00lS attended the- Propulsion Engineering 
TO Basics course prior to entry into their respective "A" schools ThPir 
records were also obtained from CMI files. ^ Llve scnoois. iheir 

GRADUATE SAMPLES. As mentioned previously, the names of school araduatP* 

C ET^^Mp 1 ^^ 1n !! e ^AVEDTRACOM TAS graduate sables" o rom 

records were obS/V^i °J ?[ aduates for each sch ° o1 for whom TAS 
records were obtained. Over all of the schools studied, a total of 7 083 

P 1 805 e rp°r ^dfr e K" 1ned f ° r the SChools shown in ihe table 

plus 1,805 records for those school graduates who first attended CMI basic 

Sruct o?uLd e T 1n V n The table als0 showfthe m th 1 f 

stSs Thl mlpff^ 5 ' 110 ? 1 and *5 e delusive graduation dates for the 
able on ;»rllhil?5j" ° graduates of each school for whom , data were avail- 
courses iSI^ILJl« inteP S t T e als ° Sh0wn - Note that some individualized 
instances thp pnf^f ^ end - of ~ c A 0 ^ ?rade while others do not. In some 
2i5 ! «* end-of -course grade is given based on a comprehensive 
examination; in others, it represents some combination of module test 
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TABLE 4. COURSE SAMPLES 



Self-Paced 


Group-Paced 


Damage" Control Ass't (Phila)l 


Damage Control Ass't (T.I.) 1 (DCA) 


Hull Technician II (Phila) (HT) 


Hull Technician I (T.I.) (HT) 


Machinist's Mate, 600 psi3 (MM6) 


Hull Technician I (Phila) (HT) ; 


Machinist's Mate, 1200 psi3 (MM12) 


Radioman, Sea2 (rm) 


Instrumentman (IM) 


Radioman, Shore2 (rm) 


Training Device Repairman (TD) 


Electrician's Mate (EM) 


Personnelman (PN) 


Fire Control Technician-(FT>- 


Yeoman (YN) 


Gunner's Mate (GM) 


Disbursing Clerk (DK) 


Av'n Support Eq: Tech (ASE) 


Aviation Storekeeper (AK) 


-Av'n ASW Operator (AW) 




Ship's Serviceman (SH) 




Aviation Electrician's Mate (AE) 


* " 

CMI 


Mixed 


Engineman3 (EN) 


Hull Technician II (S.D.) (HT) 


Av'n Machinist's Mate (AD) 


t 



^Officer Course 

2 A1 so obtained data from CMI RM^basics course 
3 Also obtained data from CMI PE basics course 
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TABLE 5. GRADUATE SAMPLES 



Course 



Type 



Original 
Graduating Sample 
4 Dates Size 



Time End of Ratings 
to' Course , on TAS 
ASVAB Complete Grade Items 



AK 


SP 


10/79-4/80 


254 


241 




n 
u 


i 7n 
1 /U 


• OCA (.Phi la) 


sp 


10/79-3/80 


60 


0 


*59 


CA 


A? 


DK 


SP 


8/79-2/80 


110 


107 


110 




7Q 

/y 


HT2 (Phila) 


SP 


10/78-3/79 


319 


318 


294 


^0^ 


c 10 


IM 


SP 


* 10/79-4/80 


18 


18 


1 7 
1 1 


10 


lb 


MM 600 


SP 


10/78-3/79 


374 ' 


372 


284 


282 


138 


MM 1200 ... 


SP 


9/78-3/79 


204 


203 


176 


176 


124 


PN 


SP 


11/78-5/79 


84 


74 


69 


> 

0 


*65 


TD 


SP 


1/79-7/79 


185 


179 




U 


1 C 1 

162 


YN 


SP 


11/78-5/79 


222 


208 


211 


0 


174 


AD 


CM I 


10/79-4/80 


464 


450 


454 


464 * 


98 


EN 


CMI 


3/79-6/79 


359 


356 

/ 


349 


244 


192 


HT2 

(San Diego) 


MIXED 


10/78-4/79 


264 


260 


261 


• 190 


.171 


AE 


GP 


3/79-5/79 


90" 


/ 86 


90 / 


90 


54 


ASE 


GP 


8/79-2/80 


36 


/ 29 

/ 

i 


36 


36 


29 



ER?C 



23 



0 
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TABLE 5. GRADUA/TE SAMPLES (continued) . 



Course Type 


Graduating 
Dates 


- < 

Original 
Sample 
Size; 


ASVAB 


Time 
to 

Complete 


End of 
.Course 
Grade 


Ratings 
on TAS 
Items 


AW' GP 


1/79-7/79 


47 


42 


45 


45" 


35 


DCA GP 10/79-4/80 . 
(Treasure Island) 


46 


0 


46 


46 


37 


EM GP 


8/78-2/79 


362 


354 ' 


349 • 


306 


262 


FT GP 


10/78-4/79 


77 


77 


66 


66 . 


■ 34 


GM GP 


7/79-12/79 


47 


47 


47 


47 


21 ' 


HT1 GP 1 
(Phi la) 


10/78-3/79 


319 


318 


319 


0 


218 


HT1 GP • 10/78-4/79 
(Treasure Island) 


264 


260- 


198 


197 


171 


RM-Sea GP 


2/79-12/79 


496 


495 


487 


0 


369 


RM-Shore GP 


3/79-8/79 


400 


397 • 


* 400 


0 


396 . 


SH GP 


10/79-3/80 


177 


176 


177 


177 


, 132 


Subtotals :SP 


10/78-4/80 


1830 


1720 


1636 


941 


1188 


CMI 


3/79-4/80 


823 


806 


803 


708 


290 


MT*ed.4Q/78-4/79 


264 


260 


261 


190 


171 


GP 


• 8/78-4/80- 


-2361 


2281 


2260 


1010 


1758 , 


Total 


8/78-4/80 


5278 


5067 


4960 


2849 


3407 



:RLC 



24 



7 "•■ 
ox 



I 



, Technical Report 117 

data Collection 

The names, graduation dates, and social security numbers of school 
graduates were obtained from NAVEDTRACOM TAS files. Visits were made to tho 

j c hoo J 5 between August and November 1980 to obtain data on course gradu- 
ates. At the schools, data were manually recorded from class records and 
!25lS?«! n work ^ e * t°r ms for subsequent entry into computer files. Data 

S2n"LX aS acc9mp J'SSS either by TAEG project staff or sch001 SMEs func- 
tioning* under genera] TAEG supervision. Information recorded consisted 

m y 2! ^-of -course grades and time-to-complete training. Where 
llrllZA 9 rl m f asures of training effectiveness/efficiency were also 
or ' 'hese other measures included numbers of academic remediations 
and setbacks, and numbers of additional hours of instruction required, 
training cost data relevant to each course were obtained from CNET. 

o^JIh 1 ! 1 " 9 H eq il% ^ ings and questionnaire task' statements "were 
obtained from the CNET TAS data base. ; The data included the fleet super- 

skill /knowledge item of the course feedback questionnaire. The 12 ASVAB 
h^: e ?L S ^ reS 4.°! eaCh ^aduate were also obtained from the CNET TAS data 
asslaned SSVft 6 ™\ m *l ter necessar ^ During school visits, 

tiSXSof^ 3 *? ? at ? ach school) classified the items on the TAS ques. 
taf? cnn • ^generic task categories. This was done under general . 
TAEG supervision and under the guidance presented in appendix C. Subject 
patter expert classifications were also entered into the data base. All 
data collection was completed by mid-1981. . 

DATA ANALYSIS 

: Analyses of training effectiveness and training efficiency were con- 
ducted on four different groupings of the courses in the study. These 
groupings are shown in table 6. / * 

■ The first analysis was based on data Of 19 single-phase enlisted "A" 
InrT l\ ? urp °^ e T f this anal ysis was to investigate possible differ- 
ences-, between CI and II across courses as 7 well as to assess interrelation- 
J p ; m w lth . the two other predictor variables (i.e., student ability level 
and type training task). < J 

Data concerning two dual-phase enlisted "A" Schools were used in the 

ItlZi T/- yS l S ' J" t - ese schools » enlisted personnel, received a different 
method of instruction in each phase of the "A" School A major purpose of 
the second analysis was to investigate the possibility of transfer effects 
between-methods of instruction within the same group of graduates. 

d«4.u T 5f. third analysis, was on data' of two single-phase officer courses. 
Both officer courses nominally present the same subject matter but each under 
a different method of instruction/ Thus, the purpose oV the third analysis 
was to investigate possible differences in methods of .instruction between 
the two officer courses, when training content is held "constant. " 

/ * * 



J 

. llf Data obtained from all the/ courses included in the previous three analyses 
■• plus the two basic, pre-"A" School courses were used for the fourth analysis. 
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/ 

... . . TABLE 6. COURSES INCLUDED IN THE FOUR ANALYSES 

/ . '• • 



First Analysis Second Analysis / ' Third Analysis Fourth Analysis 

Name N Type Name CDP N Type Name CDP N Type -Name N Type 



AD 464 CMI HT2 6339 319 SP DCA 3214 46 SP ' All courses in 
EN 359 CMI HT1 6120 319 /GP previous analyses 



/ 



AK 


254 


SP 

or 




110 


or 


TM 


1ft 


^P 
or 


MMfi 


77/1 
o/ H 


QD 
or 


vivixC 




or 


PN 
r ii 


OH 


cp 

or 


TD 

1 u 


IftR 


<;p 

or 


YN 


222 


SP 


AE 


90 


GP 


ASE 


36 


GP 


AW 


47. 


GP 


EN 


362 


GP 


FT 


77 


GP 


GM 


47 


GP 


RMSE 


496 


GP 


RMSH 


400 


GP 


SH" 


177 


GP 



plus 

DCA 3218 60 - SP • RM Basics 894 CM 

PE Basics (EN) 359 CM 

PE Basics (MM) 552 CM 



9 

ERIC 



26 



Technical Report 117 



oFtKttK! !L¥ f0urth a^^is was to provide an in-depth investigation 
of training efficiency and/or effectiveness within each course studied. 

_. Tfte present report addresses only the results of the first analvsis 

v/ - 

SS 1 ^ M 5! EL ; A P art ,! al hierarchicaTVegressibn model was employed to 
SET* rV ffe ?Sc 0f ,,? ac t set of P^ di cfors on A the criterion variables 
ti? oniJa ^ e ?h e 1 ? 7 ^i Kim * K ° h0U *' ¥ 5) - This model all °wed a un1qu"par- 
llrl VZl ! ? e t S* al vanance of ea6h criterion *f be accounted for by 
J m,if?S? e P re ^ctors entered In*? -the regression equation The use of 
approa he< Tr??^ .^/bnsonant with current methodo ogical 

1Q77? II ij investigating aptitude-treatment interactions (Cronbach & Snow 

ttrtliall! siSlfSIS* T G f Ud Jf; ej ed1 S 1 t ? r ^ iables were l cons?de?eS s?a-' 
s an 1f1caK?R< : S? \E m l6Va ?^ 1 I : Jft they met an a "eptable- level of 
of the c??ter Sn ,K ?) P^njed additional explanation of the variance 

ion variable 2 ^rS? e J inCre T t th ? R by 2 P ercent )- For each criter- 
2<Im i overall regression analysis was first performed. Then 

f ca^-on a of C S a hr°^^ re 1 made f| n the variables ' See appendix F fS ^peci^ 
fication of the hierarchical regression model used in the study. . 

JeuJSnShiS^S^h Bef ? r e Proceeding with specific investigations of 

(see appendix S). Thus, graduates foW whom data on end-of -course/ grades 
J !n"^°T lete ', and . TAS ratin 9 s were P r «ent did' not differ InTany comb ina- 
J 2w S aC ^ r0und characteristics (e. ( g., ability level by method of Sstruc- 
tion) from those graduates who had no data present on the criterion variables.. 

/ 
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SECTION III 
RESULTS 



h> ^ Th6 "!i e ! UltS ? f t he stud y are Provided in this section. All results are 
based on data analyses conducted across 19 different "Al" school s/couWas. 
nrln n^L C ° U S es "^individualized (eight SP,.two CMI) and riine we're 
?h«V?^ «J he t s ^ ec ] fi c courses examined were identified instable 6 (under 
tha\colunin "first analyse"). . The-original sample sizes (N) of" graduates 
trained" under each instructional method are given in table 7. 

TABLE 7. ORIGINAL SAMPLE SIZES OF GRADUATES'" 



Method of Instruction 


N 




< 


Individualized fSP + cmi)~ 




2274 




Self-Paced (SP) 


-J451 






Computer Managed Instruction (CMI) 


' 823 






Conventional (GP) 




1732 




Group-Paced (GP) 


* 

1732^- 






.Total 




4006 





liea , In J; hls sectl on. a relatively large number of tables and figures are 
used. These are necessary to display adequately the study findings and to 
present the supporting statistical evidence for the many significant 
relationships observed among the variables of interest. All significant ■ 
findings are shown, however, in three tab les„ presented at the end of the 
section (taWes 35y36, and 37). In the main body'of the section, the 
results are organised according to the format described immediately below. 

All results are presented in order of the criterion variables examined; 
i.e., the measures of training effectiveness and training efficiency. For 
each criterion measure, tables are presented to summarize the .siqnifi cant 
results, of overall regression analyses. Each summary table for a* criterion 
measure identifies the predictors assessed and lists the regression coeffi- 
cients (B) and F-ratios (F) obtained. The summary tables also show the 
degrees of freedom (DF) involved in specific comparisons and the magnitude 
ot a particular R* increment. The numbers listed under *'step" in the sum- 
mary tables refer to the order in which that particular set of predictor 
variables was entered into the regression equation (see appendix F). 

As mentioned abdve, the statistical summary tables identify the 
significant results Relative to a given criterion variable. After the 
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summary tables, each significant predictor, both main effects and .interactions, 
is further delineated separately. Supporting data for the significant results 
are presented in tables. For each criterion variable, nonsignificant results 
are grouped together in one paragraph. Note that in the main effects analyses, 
data concerning a particular criterion variable are combined over" all courses 
and over all graduates. The purpose of the main effects analyses is to assess 
tfie contribution to criterion variance of one particular predictor variable 
(e.g., method of instruction, trainee ability). 'For the present study, the 
interpretation of significant main effects should be tempered by a significant 
interaction of that main, effect with another predictor variable. 

TRAINING EFFECTIVENESS MEASURES 

The criterion measures of training effectiveness were end of course 
grades and TAS ratings. Two variables were formed from the TAS ratings: 
overall mean TAS ratings and task -specific TAS ratings. 

CRITERION: END-OF-COURSE GRADES. End-of -course grades represent the average 
percent correct obtained by a graduate on items tested in the school. End- 
of-course grades obtained from the schools were based on either combinations 
of scores from module/lesson examinatipn5.administered during the course of 
training, or on comprehensive examinations administered at the end of training. 
The overall results of regression analyses of end r 6f -course grades are presented 
in table 8.* Significant and nonsignificant results are further detailed 
below. ■ ' _ . 

Method of Instruction. Graduates of individualized courses (SP + CMI) received 
higher end of course grades than CI (GP) graduates. SP graduates received higher 
end of course grades than CMI graduates (see table 9). 

Ability Level . Across all II and CI courses, graduates with higher AFQT "percen- 
tile scores received higher end of course grades than those with lower scores 
(see table 10). . . . 

Method of Instruction by Ability Level. Individualized instruction-(SP + CMI) / 
•graduates at the higher AFQT percentiles received higher end of course grades 
than CI (GP) graduates at the higherpercenjtiles. Lower ability II and CI gradu- 
ates received similar end of course grades.. Conventional instruction graduates/ 
. at higher AFQT percentiles received end of course grades similar to those of CI 
graduates at the lower AFQT percentiles. For both SP and CMI graduates, AFQT was 
positively related to end of course grade..! Both SP and CMI graduates had similar 
positive relationships between end of course grades and ability level (see figure 
1 and table 11). ' * 

Training Task . Across II and CI courses, end of course grades were higher in 
courses that taught a larger percentage of category, procedure, and rule tasks 
than in courses that taught a lesser percentage of these tasks. End of course 
grades were higher in II and CI courses that taught a smaller percentage of ract 
and principle tasks than in courses that taught a larger percentage of these. . 
tasks (see table 12). 



ERIC . 



\\ 30 on 



~ • Technical Report 117 

TABLE 8. SIGNIFICANT RESULTS FROM REGRESSION ANALYSES OF END-OF-COURSE GRADES 



Step Variables 


B 


F'r 


DF 


R2 

Increment 


Overall F 


DF 


1 il vs CI (Tl) 


-4.13 


376.72 


1/2041 


.2034 


260.64 


2/2041 


5P vs CMI (T2) 


-2.55 


169.41 


1/2041 




* 




2 AFQT (A) 


.13 


■ 

188.95 


1/2036 


- 

.2041 


140.29 


5/2036 


Fact - (11) 


-.60 


175.31 


" 1/2036 








uategory ( iz J - 


. .28 


151.5.8 


1/2036 








Procedure (13) 


.13 ' 


141^29 


1/2036 








Rule- (14) 


.10 


92.31 


1/2036 






r 


> 

. 3 Tl x 11 


2.27 


• 

49.73 


1/2031 


. 1 

\ 

.0831 


66.27 


5/2031 


Tl x 13 


-.30 


133.95 


1/2031 








A x Tl 

< 


-.04 


13.07 


1/2031 
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TABLE 9. END-OF-COURSE GRADES BY METHOD OF INSTRUCTION 



Method of Instruction 


Mean 


SD* 


. N 


SP 


90.84 


5.51 


584 


CMI 


85.74 


8.24 


- 692 


II (SP +.CMI) 


88.08 * 


7.56 


1276 


CI (GP) 


82.10 


6.72 

•* 


768 



♦Standard Deviation 



TABLE 10. END-OF-COURSE GRADES BY AFQT PERCENTILE RANGE 
AND MENTAL: CATEGORY 



AFQT 

Percentile Range 


Mental Category Mean 


sp 


N 


93-99 


1 ; 


92:68 


6.51 


135 


65-92 


2 ' : 


87.45* 


7.63 


673 . 


49-64 


3U 


84.55 


7.32 


737 


31-48 


3L 


. 83.58 


7.63 


436 


21-30 


41) 


; 81.87 


7.83 


21 


10-20 


4L 


79.50 


o.'o 


1 



t 
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MEAN 
END-OF- 
COURSE ■ 
•GRADES 



95 



90. 



85 - 



80 



II (SP + CMI) 




1 



4U " 3L 3U 2 

MENTAL CATEGORY . 

4 

21-30 31-48 I 49-64 65-92 
AFQT PERCENTILE RANGE 



J L 



93-99 



Note: Mean data points based on less than five cases are not plotted. 



Figure 1. Mean End-of-Course Grades by Method of Instruction 
and Ability Level 
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TABLE 11. END-OF-COURSE. GRADES 3Y METHOD OF INSTRUCTION AND ABILITY LEVEL 



AFQT 

Percentile 
Range 


Mental 
Category 


"of 

Instruction * 


Mean 


SD 


N 


93-99 


1 ' 


SP 


"^95.50 


3.04 


102 






CMI 


92.69 


9.16 


2 






SP + CMI 


95.44 


3.17 


* 104 






GP 


83.41, 


6.30 


31 


65-92 


2 


SP 


91.96 


4.95 


273 






CMI 


90.38 


6.13 


140 






SP + CMI 


91.42 


5.43 


"3 






GP 


81.33 


6.24 


260 


49-64 


3U 


SP 


87.29 


4.64 


146 






CMI 


86.42 


7.99 


< 

263 






SP + CMI 


86.73 


6.98 


409 






GP 


81.83 


6.80 


328 


31-48 


3L 


SP 


86.38* 


4.79 


58 






CMI 


82.81 


8.27 


253 




■ 


-SP + CMI 


83.47 


7.86 


311 


t 




GP " 


83.84 


7.05 


125 


21-30 


4U 


SP 


0 


n 


u 






CMI 


82.12 


7.95 


20 






SP + CMI 


82.12 


7.95 


20 






GP ' . 


77.07 


0 


1 


10-20 


4L 


GP 


79.05 


0 


1 
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TABLE 12. END-OF-COURSE GRADES BY PERCENTAGE OF TASK- TRAINED 



Task 


Percent 


Mean 


SD " 


'• N 


Fact 


1-15 
16-30 


81.35 
77.69 


6.66. 
4.79 


852 
66 


Category 


, 1-15' 
16-30 


84.92 
89.02 


5.32 . 
5.52 , x 


859 

680- ' 


Procedure 


1-50 
51-100 


82.94: 
87.41 


5.03 * 
6.30 


721 
1323 


Rule 


1-50 
51-100 


82.47 
86.08 


6.21 
4.89 


1237 
110 


Principle 


1-50 
51-100 


87.98 
79.36 


4.82 
5.35 


936 
306 



Method of Instruction by Training Task. Individualized instruction graduates 
received higher grades in courses that taught a smaller percentage of fact tasks 
than CI graduates in courses that taught a -smaller percentage of fact tasks. 
There were no data points from II courses for a larger percentage of fact tasks 
on which to base a comparison with CI courses. Individualized instruction 
graduates received higher grades in courses that taught a larger percentage of 
procedure tasks than II graduates who were taught a smaller percentage of 
-procedure tasks. Conventional instruction graduates received similar grades in 
courses that taught smaller and larger percentages, of procedure tasks, (see fiqure 
2 and table 13). / ' 



TABLE 13. END-0F-C0URSE GRADES BY PERCENTAGE OF TYPE OF 
TRAINING TASK AND METHOD OF INSTRUCTION 
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Task 


Percentage 
Range 


Method 


Mean 


SD 


N 


Fact 


1-10 


II 


83.08 


7.65 


453 






CI 


79.39 


5.44 


399. 




11-20 


II 


o • 


0 


0 






CI 


77.69 


4.80 


66 


Procedure 


1-50 


II 


85.65 


5.07 


126 






CI 


82. 37 


5.11 


595 




51-100 


II 


88.34 


7.04 


1150 






CI 


81.19 


6.17 


173 
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n 









FACT 




PROCEOURE 


MEAN 


90 








II (SP + CMI) 


tnu-Ur- 
COURSE 


85 




X 






GRADES 


80 






U (GP) 
1 | 








1-10 


11-20 


1-50 . 51-100 



• PERCENTAGE OF TASK TAUGHT 

I 



Note: Mean data points based on less than five cases are not plotted. 



Figure 2. Mean End-of-Course Grades by Method of 
Instruction and Training Task 
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Nonsignificant Results. The interactions of ability level by training task, 
and ability level by training task by method of instruction did not signifi- 
cantly predict end-of-course grades. The main effects for sex of graduate 
and geographic location of school were also not significant. 

CRITERION: OVERALL TRAINING APPRAISAL SYSTEM (TAS) RATINGS. For this 
analysis, a mean (overall) TAS rating was computed for each course graduate. 
The means were derived from supervisor ratings of the graduate on individual 
TAS questionnaire items. Significant results obtained from the overall 
regression analyses are presented in table 14. Specific findings are listed 
below. 



TABLE 14. SIGNIFICANT RESULTS FROM REGRESSION ANALYSES OF OVERALL 
TRAINING ADEQUACY RATINGS . 



Step 


Variables 


B 


F 


DF " • 


R2 

Increment 


"Overall F 


' DF 


1 


II vs CI (Tl) 


-0.16 


62.36 


1/2412 


.0267 


33.10 ' 


2/2412 


2 


Fact (11) 


0.01 


.4.23 


1/2407 


.0378 . . 


19.46 


5/2407 




Category (12) 

i 


-0.02 ■ 


81.74 


1/2407 









Method of Instruction . Individualized instruction (SP + CMI) graduates received 
higher mean TAS ratings than CI (GP) graduates. There were no significant 
differences between SP and CMI graduates on mean TAS ratings (see" table 15). 



TABLE 15. OVERALL TAS RATINGS BY METHOD OF INSTRUCTION 



Method of Instruction 


Mean 


SD 


N 


SP 


2.90 


.70 


899 


CMI 


2.96 


.57 


287 


II (SP+CMI) 


2.91 


.67 


1186 


CI (GP) 


2.69 


.67 


1229 . 
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Training Task . Across II and CI courses, graduates who attended those courses 
that taught a smaller percentage of fact or category tasks received higher 
average TAS ratings than graduates who were taught a greater percentage of 
.these tasks (see-taWe-l^ x 

/ TABLE 16. TAS RATINGS BY TYPES OF TASKS" 



Task 


Percentage > 
• . of Task 
Taught 


Mean 


SD 


N 


Fact 


1-10 


2.86 


.56 


408 




11-20 


2.73 


.64 


34 


Category 


1-10 - 


2.92 


.66 


1027 




" 11-20 


2.66 


.69 


1149 




21-30 


2.73 


.64 


34 



i 



Nonsignificant Results . All interactions involving combinations of ability 
level, method of instruction, and training task did not significantly predict 
mean TAS ratings. The following main-effects were, also nonsignificant: ability 
Jevel, end-of-course grade-, Ume-to-complete the course, *sex of graduate, 
and geographic location of school. 

CRITERION: TASK-SPECIFIC TAS j RATI N6S. TAS ratings were also grouped by 
items classified into generic tasks (i.e., facts, categories, procedures, 
rules, and principles). Then, mean TAS ratings were- computed for each of 
the generdc task groupings. These groupings and! resulting means permitted 
task-specific TAS ratings to be regressed on a specific task, and on ability 
level, method of instruction, and their interactions. One purpose of this 
analysis was to determine if the percentage of^a' particular task trained in 
"A" School was related to the ratings obtained on TAS items classified as 
that particular task. This permitted answering questions of the form, "Are 
higher TAS ratings on items classified as, e.g. fact tasks, associated with 
a larger percentage of, e.g. fact tasks, taught in 'A 1 School?" This analysis 
would clarify if specific training content in v the school setting transfers 
to. the fleet (operational) setting (as reflected in the task-specific TAS 
ratings). Another purpose was to assess the interactions of specific tasks 
with trainee ability level and-method „of instruction. 

Significant results of the regression analyses for all task-specific TAS 
ratings are presented in table 17. Then, the results of these analyses are 
further delineated below according to the generic task to which the TAS ratings 
apply (criterion variables). 

Criterion: Fact TAS Ratings . Percentage of fact task trained, AFQT 
percentile, method of instruction, and the interactions of these variables 
did not significantly predict fact TAS ratings. 
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TABLE 17. SIGNIFICANT RESULTS FROM REGRESSION ANALYSES OF 
TASK-SPECIFIC TAS RATINGS 

ft 



Criterion 


Step 


Variable 


B 


> 


OF 


R2 

Increment 


Overall F 


DF 


Category 


2 


Category (12) 


-.02 


55.20 


,,1/1703 


.0309 


27.58 


2/1703 


Procedure 


3 


Tl x 13* 


-.01 


75.53 


1/2336 


.0334 


27.73 


3/2336 


-Rule 


3 


Tl x 14** 


-.02 


31.20 


1/228 


.0297 


■ 12.64' 


3/1228 






A x T2*** 


.01 


5.01 


1/228 






Principle 


1 ■ 


Tl**** 


' .15 


19.14 


1/871 


.0215 


19.14 


1/871 



' *T1 x 13 « Method of instruction by procedure task interaction. 
* **T1 x 14 = Method of instruction by rule task interaction. 
***A x T2 = AFQT by method of instruction (SP vs CMI) interaction. 
****T1 = II vs CI. 



Note: No significant results for fact TAS ratings. 



Criterion: Category TAS Ratings . 

Percentage of Category Task Trained.- Across II and CI courses, graduates 
who were presented with~a smaller percentage of category tasks during train- 
ing received higher category-specific TAS ratings than graduates presented 
with a larger percentage of category tasks during training (see table 18). 



TABLE 18. CATEGORY-SPECIFIC TASK RATINGS BY 

PERCENTAGE OF CATEGORY TASKS TRAINED 



Percentage of 








Category Task 


Mean. 


SD 


N 


1-15 


3.10 


.63 


■1055 


16-30 


2.70 


.63 


583 



.Nonsignificant Results. AFQT percentiles, method of instruction, and their 
interactions with percentage of category task trained did not significantly 
predict category TAS ratings. 
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Criterion: Procedure TAS Ratings . 

Method of Instruction by Percentage of Procedure Task Trained. Conventional 
instruction graduates who were presented with a 1 smaller percentage of proce- 
dure tasks during training received higher procedure-specific TAS ratings 
than II graduates presented with a smaller percentage of procedure tasks. 
On the other hand, II graduates who were presented with a larger percentage 
Of procedure tasks during training received higher procedure-specific TAS 
ratings than CI graduates presented with a larger percentage of procedure 
tasks during training (see figure 3 and table 19). Percentage of procedure- 
task taught. was positively related to procedure TAS ratings for II graduates 
and negatively related for CI graduates. There were no significant differences 
between SP and CMI graduates on procedure TAS ratings. 




-J 1 

1-50 51-100 

* 4 

PERCENTAGE OF PROCEDURE TASK TAUGHT 



Figure 3. Mean Procedure TAS Ratings by Percentage of Procedure 
Task Taught and Method of Instruction 
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TABLE 19. PROCEDURE-SPECIFIC TAS RATINGS BY PERCENTAGE OF 

PROCEDURE TASK- TRAINED AND METHOD OF INSTRUCTION 



Percentage of 
Procedure Task 

r 




II 






CI 




Mean ' 


SD 


N 


Mean 


SD 


N 


1-50 


2.71 


.70 


209 


' 3.00 


.73 


462 


. 51-100 


2.95 


.68 


920 


2.55 


t 

.63 


753 
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Nonsignificant Results. AFQT percentile/ percentage of procedure tasks 
trained, ^method of instruction, and their remaining interactions did not 
significantly predict procedure-specific TAS ratings. 

Criterion: Rule TAS Ratings . 

, Method of Instruction by Percentage of Rule Task Trained. Individualized ' 
instruction graduates who were presented with a smaller percentage of rule 
tasks during training received higher rule-specific TAS ratings than CI gradu- 
ates presented with a smaller percentage of rule tasks during training. On 
the other, hand, CI graduates who were presented with a larger' percentage of 
rule tas^s during training received higher* rule-specif ic TAS ratings than II 
graduates presented with 'a larger percentage of rule tasks during training. 
Percentage of rule task was negatively related, to rule TAS ratings for II 
graduates and positively related for CI graduates (see figure 4 and table 
20). / There were no significant diffe. ences between SP and CMI graduates on 
* Rule/ TAS ratings. » • 

/ 

Method of Instruction by Ability Level. Higher ability CMI graduates received 
higher rule-specific TAS ratings than higher ability SP and GP graduates. 
Average to lower ability SP and GP graduates received rule-specific TAS ratings 
^similar to those received by average to lower ability CMI graduates. There 
were no significant differences between II (SP and CMI combined) and CI (GP) 
/ graduates on rule-specific TAS ratings by ability level (see figure 5 and 
I table 21). 

Nonsignificant Results. Percentage of training task, AFQT percentiles, 
method of instruction, and .their remaining interactions did not significantly 
predict rule-specific TAS ratings. 

• Criterion: Principle TAS Ratings . 

Method of Instruction. Conventional instruction (GP) graduates received 
higher principle- specific TAS ratings than II (SP and CMI) graduates. 
There were no significant differences between SP and CMI graduates on 
principle-specific TAS ratings (see table 22). " 

Nonsignificant Results. Percentage of principle tasks trained, ability 
level, and their interactions did not significantly predict principle- 
specific TAS ratings. 
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MEAN 

RULE 
TAS. - 
RATING 



3.1 h 

3.0 
2.9 - 

2.8 

2.7 
2.6 
2.5 - 



X 



CI (GP) 




s 

II (SP + CMI) 



1 



1-40, 41-80 
PERCENTAGE OF RULE TASK TAUGHT 



Figure - 4. Mean Rule TAS Rating by Percentage of Rule 
Task Taught and Method, of Instruction 



/ 



TABLE 2(U RULE-SPECIFIC TAS RATINGS BY PERCENTAGE OF RULE TASK 
TRAINED AND METHOD OF INSTRUCTION 



" ■ /Percentage- of 
/Rule Task 




' II 






CI 




Mean 


SD 


N 


Mean 


SD 


N 


/ 1-40 


3.00 


,80 


450 


2.72 


.67 


i 

417 


1 41-80 


. 2.80 


.90- 


239 


3.17 


.52 


130 



/ ; 
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3.5 



MEAN 
RULE 
TAS 

RATING 



3.0 



2.5 




N .' 



.4U 



. 3L 



3U 



MENTAL CATEGORY 
21-30 31-48. 49-64 * 65-92 

AFQT PERCENTILES 



93-99 



Note: Mean data points based on less than five cases' are not plotted. 



Figure 5. Mean Rule TAS Rating by Ability Level and 
Method of Instruction. 
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TABLE 21. RULE-SPECIFiC TAS RATINGS BY ABILITY LEVEL 



AND ME' 


HOD OF INSTRUCTION 


AFQT 

Percentile 


Mental 
Category 


Mean 


k SP 
SD 


• 

N 


- 

Mean 


CM I 

SD 


N 


Mean 


GP 
SD 


N 


93 : 99 


1 


2.84 


.70 


47 


3.45 


.93 


u. 


2.89 


.72 


11 


65-92 


2 


2.87 


.74 


220 


3.10 


.78 


15 


2.75 


.73 


188 


49-64 


3U 


2.93 


.74 


218 


2.8F 


.81 . 


24 


2.84 


.70 


235 


31-48 . 


3L 


2.92 


.86 


107 


3.0 




1 


,-2.86 


.72 


92 


21-30 


. 4U 


2.93 


.45 


7 








3.0 




1 


10-20 


4L 














4.0- 




1 



0 

TABLE 22. PRINCIPLE-SPECIFIC TAS RATINGS 
BY METHOD OF INSTRUCTION 



Method of 








Instruction 


Mean 


SD 


N . •<" ' 


II (SP + CMI) 


2.59 


.87 


384 


CI (GP) 


2.82 


.67 


489 



TRAINING EFFICIENCY MEASURES 

Thp measures of training efficiency examined were time t~ complete the 
course and training costs. The results obtained concerning training 
efficiency are presented below. 

CRITERION: TIME TO COMPLETE THE COURSE. Time to complete the course was 
represented by student contact hours* Student contact hours reflect the 
amount of time the student was under direct instruction. Significant 
results of the regression analyses performed are summarized in table 23. 
The results are further detailed below. 



9 

ERLC 



44 



.0 



Technir all Report 117 



TABLE '23. SIGNIFICANT RESULTS FROM REGRESSION ANALYSES OF 
TIME TO COMPLETE (CONTACT HOURS)" COURSES 



Step 


0 

Variables 


B 


/ 

I 

""7* 


OF 


R2 

Increment 


Overall f_ 


OF 


1 


II vs CI (Tl) 
SP vs CMI (T2) 


28.14 
-13.34 


157/41 
33,189 

I 


1/3789 
1/3789 


.0436 


86.30 


2/3789 



AFQT (A) - 0.70 

Fact (II) 12.89 

Category (12) - 5.96 

Procedure (13) - 1.81 

Rule (14) - 0.79 



73.86 
700.34 

1Q12.49 
536.40 

/ 67.98 



1/3784 
1/3784 
1/3784 
1/3784 
1/3784 



.4903 3979.38 



Tl x II 
Tl x 12. 
Tl x 13 
Tl x 14 
A x-Tl 
A x T2 



27.47 
9.56 / 
3.42/ 
3.9J 
l.GO 
0.47 



318.38 
304.58 
268.62 
311.11 
85.24 
19.69 



1/3778 
1/3778 
1/3778 
1/3778 
1/3778 
1/3778 



.0659 



163.67 



5/3784 



6/3778 I 



Method of Instruction . Individualized instruction (SP + CMI) graduates com- 
pleted .their courses in less time than CI (GP) graduates. Within the II 
group, CMI graduates completed their courses in less time than SP Graduates 
(see'table 24). 3 

TABLE 24. TIME TO COMPLETE THE COURSE (CONTACT HOURS) 
BY METHOD OF INSTRUCTION 



MethcJ of Instruction 


Mean 


SD 


N 


SP 


166.35 


69.60 


1283 ■ 


CMI 


139.57 


88.25 


803 


II (SP+CMI) 


156.04 


82.57 


2086 


CI (GP) 


195.22. 


122.05 


1707 



Ability Level. Across II and CI courses, graduates with higher AFQT percen- 
tile scores finished their courses in less time than those with lower AFQT 
percentile scores (see table 25). Graduates at the mid-AFQT percentiles had 
similar course completion times. These main effect findings are tempered, 
however, by the interaction analysis given below. 



9 

ERIC 



45 . 

51 



Technical Report 117 



TABLE 25. TIME TO COMPLETE THE COURSE (CONTACT HOURS) 
BY ABILITY LEVEL ' 



AFQT 
Percentile Range 


Mental Category • 


Mean 


SD 


N 


.93-99 


1 


134.64 ' 


93.05 


203 


65-92 


2 


178.19 


il3.29 


1114 


49-64 


3U 


174.11 


103.45 


1551 


31-48 


3L 


172.12 


91.30 


800 


21-30 


4U 


,191.73 


83.40 


34 


10-20 


4L 


480.00 


0.0 


1 



Method.of Instruction by Ability Level. Individualized instruction graduates 
in the upper mental categories finished their courses in less time than cor- 
responding CI graduates. Both II (SP + CMI) and CI (GP) graduates in the 
mid and lower mental categories took about the same amount of time to complete 

2?IU°l! rSe J-5?! e t 19U C e 6:a ? d table 26) * Traini "9 tine was negatively 
related to ability level for II graduates, and positively related for CI 
graduates. Training time was significantly less for CMI graduates than for 
graduates at each ability level except for the lowest ability level. SP 
S;f e K s . ?{ significantly less training time than CMI graduates at the 
lowest ability level (menial category 4U). ; 

Training Task , Across II and CI courses, student completion times were longer 
for courses that taught a greater percentage of fact or principle tasks - 
than a lower percentage of these tasks. Student completion times were shorter 
in ; courses that taught a lower percentage of category, principle, or rule 
tasks than a higher percentage of these tasks (see table 27). 

Method of Instruction by Training Task. Individualized instruction graduates 
had lower completion times in courses that taught a smaller percentage of 
I a c5;iw e90ry ' ; roced " re .' ™le, or principle tasks than CI graduates taught 

JeSllrfi^w* - 9e ° f th6S 2u taS kS * u In additi ™> " graduates had lower . 
completion times in courses that taught a larger percentage of category or 
procedure tasks than CI graduates taught a larger percentage of these tasks. 
For II graduates, course completion time- was negatively related to the per- 
centage of category or procedure tasks taught. There was no significant 
relationship between course completion time and percentage of rule tasks 
taught for II graduates. For CI graduates, course completion time was posi- 
tively related to the percentage of fact tasks taught, but' negatively related 
t bl 28) rC 96 category ' P rocedur e, or principle tasks (see figure 7 and 
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Note: Mean data points based on less than five graduates are not plotted. 



Figure 6. Mean Time to Complete the Course by Method of 
Instruction and Ability Level 
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TABLE 26. TIME TO COMPLETE (CONTACT HOURS) BY METHOD OF 
INSTRUCTION AND ABILITY LEVEL 



Method - 

AFQT Mental of 

Percentile Category Instruction Mean SD N 





i 

I 


CD 

or 


101.50 


48.72 


152 






PMT 


OO 00 

o3.33 


OO 

38.67 


6 






or t cni 


1 no qo 
lUU.oo 


il o OO 

48.39 


158 






6P 


253.31 


113.53 


45 


65-92 


0 

c 


CD 

or 


1 AO CO 

149.5Z 


81.99 


508 


! 






115.00 


^ 1 OO 

61.08 


167 






CD + PMT 

or • Lril 


1/11 i/i 
141 ,14 


78.71 


675 






GP 


235.16 


133.08 


439 


49-64 


3U 


SP 


199.59 


ft? 11 

OL» OO 








CMI 


130.13 


67.02 


310 






SP + CMI 


168.83 


83.37 


700 






BP 


v 178.45 


117.29 


851 


31-48 


3L 


SP 


198.59 


104.86 


178 






CMI 


161.25 


69.87 


281 






SP + CMI 


175.73 


86. 98, 


459 






GP 


167.25 


96.73 


341 


21-30 


4U 


SP 


155.20 


43.06 


10 






. CMI 


188.53 


77.69 


21 






SP + CMI 


177.78 


69.51 


31 






GP 


335.87 


90.18 


3 


10-20 " 


4L 


GP 


480.0 . 
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Figure 7. Mean Time to Complete the Course by Method of 
Instruction and Type Training Task 
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TABLE 27. TIME TO COMPLETE THE COURSE (CONTACT HOURS) BY PERCENTAGE OF TRAINING TASK 



Percentage 
Ganges of 
Task Taught 




Fact 




Cateaorv 






Procedure 




ku ie 




Principle 


Mean 


SD 


N 


"Mean SD 


N 


Mean 


SD 


N 


Mean 


SD 


N 


Mean SD N 


' i in 


90C CO 
COD •DO 


11. /o 


one 

896 


232.77 81.94 


1353 


< 


. 'N/A 




245.83 


124.71 


874 


207.62 70.10 398 


n on 


aoo no 


50.53 


66 


108.45 68.75 


1797 


257.30 


120.27 


407 


267.93 


46.90 


396 


130.61 78.83 507 


21-30 




N/A 




428.03 50.53 


66 


259.46 


43.45 


349 


221.58 


88 61 


14fi 


aoq no cn co cc 


31-40 




N/A 




N/A 






N/A 






N/A 




446.67 14.99 90 


41-50" 




N/A 




N/A, 




274.11 


116.30 


194 


240.00 


0 


177 


465.29 122.23 63 


51 -60 




N/A 




N/A 




406.92 


56.60 


137 


•> 


N/A 




259.46 43.45 349 


61-70 




N/A 




N/A 




142.36 


65.81 


691 




N/A 




N/A 


71-80 




N/A 




N/A 




116.65 


23.61 


565 


196.12 


74.67 


295 


N/A 


81-90 




N/A 




N/A 




125.24 


66.. 10 


1413 




N/A 




N/A 


91-100 




N/A 




N/A 




368.00 


0 


36 




N/A 




N/A 
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TABLE 28. TIME TO COMPLETE (CONTACT HOURS) BY PERCENTAGE OF 
TASK TAUGHT AND METHOD OF INSTRUCTION 



Task 


Percentage of 
Task ! aught 




II 






CI 




Mean 


ou 




Moan 
Mean 


SD- 


N 


Fact 


1-15 


190.12 


50.46 


454 


282.16 


39.84 


442 




16-30 


0 • 


0 


0 


428.03 


50.53 


66 


Category 


1-15 


160.71 


43.15 


1197 


207.77 


55.56 


1062 


16-20 


73.84' 


13.46 


348 


163.14 


16.70 


609 


Procedure 


1-50 • 


219.72 


45.39 


312 


281.97 


33.52 


638 




51-100 


131.86 


36.94 


1773 


143.45 


10.31 


1069 


Rule 


1-50 


194.72 


52.42 


982 


306.90 


33.13 


811 




51-100 


196.12 


35.28 


295 


0. 


0 


0 


Principle 


o 

1-50 


133.34 


43.00 


662 


342.73 


22.38 


- 462 


51-100 


0 


0 


0 


259.46 


43.45 


349 



Nonsignificant Results . The interactions of ability level by training task, 
and ability level by training task by method of instruction; main effects of 
sex of graduate and geographic location of school did not significantly pre- 
dict time to complete the course. 

CRITERION: TRAINING COSTS. To conduct analyses of instructional costs, an 
individual training cost was computed for each graduate. This was derived 
by dividing a graduated course completion time by average course completion 
time and then multiplying this quotient by the "average course training cost 
to produce one graduate per course session" (see appendix C). Computing 
individual training costs in this manner provided the variance needed within 
courses to conduct regression analyses with training costs as the criterion 
variable. If agraduate's course completion time was longer than the average 
course completion time, then his/her training, cost would be greater than the 
average course training cost. If a graduate's course completion time was 
shorter than the average course completion time, then his/her training cost 
would be less than the average course training cost. Thus, the computation 
of individual training costs associated shorter course completion times for 
a graduate with less training costs. 

The overall results of the regression analyses of training costs are 
presented in table 29. Significant and nonsignificant results, both main 
effects and interactions, are listed below. 

Method of Instruction . Significantly higher training costs were associated 
with II (SP + CMI) graduates' than with CI (GP) graduates. Training costs 
were significantly greater for SP graduates than for CMI graduates. Train- 
ing costs were similar for CI (GP) and CMI graduates (see table 30). The- 
apparent contradiction of higher training costs and lower training times 

51 



Technical Report 117 
TABLE 29. SIGNIFICANT RESULTS FROM REGRESSION ANALYSES OF TRAINING COSTS 



.> l.rp 


1 rrii M M)i 


B 


F 


OF 


R2 

Increment 


Overall F 


OF 


1 ■ 


II vs CI (Tl) 


-115.97 


18.64 


1/37891 


.0293 


57.17 


2/3/89 




SP vs CMI (T2) 


-246.45 


• 80.70 


1/3789 








2 


AFQT • (A) 


- 10.85 


98.38 


1/3784 


.3753 


471.98 


5/3784 




Fact (11) 


109.08 


275.09 


1/3784 










Category (12) 


- 58.05 


526.27 


1/3784 






• 




Procedure (13) 


- 24.98 


561.56 


1/3784 










Rule (14) 


- 15.57 


146.52 


1/3784 








3 


A x Tl 


13.94 


97.70 


1/3778 


.1177 


153.10 


1/3778 




A x T2 


4.75 


11.9.4 


1/3778 










Tl x 11 


233.45 


134.60 


1/3778 










Tl x 12 


132.12 


340.22 


1/3778 










Tl x 14 


23.22- 


63.49 


1/3778 








4 


TICOM* 


11.75 


15418.92 


1/3769 


.3819 


15419.18 


1/3769 



*Time to complete the course- 
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being associated overall with 1 1, graduates is further explained by the 
interaction of method of instruction by ability level. 



TABLE 30. TRAINING COSTS BY METHOD OF INSTRUCTION 



.Method of Instruction 


Mean 


SD 


N 


SP 


$ 2573.71 


1102.36 


1283 


CMI 


2080.81 


705.78 


802 


II (SP + CMI) 


2384.11 


969.21 


2085 


CI (GP) 


2153.31 


1467.29 


1707 



Ability Level. Across II and CI courses, training costs were signif icant/iy 
greater for lower mental category graduates than for higher mental category 
graduates (see table 31). Training costs were similar for graduates at /the 
middle mental categories. This main effect is also partitioned further/by 
the interaction of method of instruction by ability level. 



TABLE 31. TRAINING COSTS BY ABILITY LEVEL 



AFQT Percentile 
Range 


Mental 
Category 


Mean 


SD 


N 


93-99 


1 


$1982.57 


1056.01 


203 


65-92 


2 


2370.01 


1285.24 


1114 


49-64 • 


3U 


2207.48 


1253.92 


1551 


31-48 


3L 


2285.47 


1136.47 


800 


21-30 


4U 


2837.38 


945.76 


34 


10-20 


4L 


4704.67 


0 


1 



Method of Instruction by Ability Level. Training costs were significantly 
greater for mid to lower mental category II (SP + CMI) graduates than for 
mid to lower mental category CI (GP) graduates. On the other hand, training 
costs were significantly greater for upper mental category CI (GP) graduates 
than for upper ^mental category II (SP + CMI) graduates. Also, training 
costs were significantly greater for mid to lower mental category SP 
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graduates than for mid to lower mental category CMI graduates. Training 
costs were similar for upper mental category SP and CMI graduates. Training 
costs were negatively related to ability level for 1 1. graduates and positively 
related for CI graduates (see figure 8 and table 32). 

The explanation for the apparent contradiction— higher training costs 
but shorter trailing time for II graduates when longer training time is 
related to higher training costs over all courses— is inherent in the nature 
of the training cost yariables used.- When training costs* are computed for 
individual graduates across all courses, students who take more time to com- 
plete their courses do indeed cost more to train than those who take less 
time to complete the course. 4 This is due to the relationship of the average 
course costs being multiplied by the graduate's individual training time 
divided by the average course completion time.,: The result of^this individual 
training cost metric is that slower students are assigned a training cost 
value greater than the average course cost and faster students are assigned 
a value lower than the average course cost. Thus; v the individual training 
costs as computed in this study, allow one to observe a* positive relationship 
overall between training time and'training cost. However, the average course 
costs represent given values from the CNET Per Capita Cost Data Base and, as ' 
such, may or may not be as~dependent on course completion time as the indi- 
vidual training costs. The higfier^costs observed for the II courses in this 
study may be due to a course characteristic^diff erence that may or may not 
be related to method of instruction. One way to clarify this issue would be 
to obtain and analyze the cost data of particular categories that make up 
the total training costs for each course in the study. The results of this 
investigation could reveal the relative contributions of instructional method, 
specific course cost categories, and their interactive effects to the total 
training costs. . 

Type Training Task . Across II and CI courses, training costs were signifi-* 
cantly higher for courses that taught a greater percentage of factor 
principle tasks than in courses that taught a lower percentage of fact or 
principle tasks. Training costs were significantly higher in courses that 
taught a smaller percentage of category, procedure, or rule taste than in 
courses that taught a larger percentage of category, procedure, or rule tasks 
(see table 33). 

Method of Instruction by Training Task. Individualized instruction graduates 
had lower training costs in courses that taught a smaller percentage of fact, 
rule, or principle tasks than CI graduates in courses that taught a smaller 
percentage of these tasks. Individualized instruction graduates had higher 
training costs in courses that taught a smaller percentage of category ta§ks 
than CI graduates in courses that taught a smaller percentage of category"' 
tasks. II graduates had lower training xosts in courses that taught a larger 
percentage of category tasks than CI graduates in courses that taught a lower 
percentage of category tasks. Training costs were not significantly related 
to the percentage of procedure tasks taught by method of instruction (see 
figure 9 and table 34). 
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Figure 8. Mean Training Costs By Method of Instruction 
and Ability Level 
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TABLE 32. TRAINING COSTS BY METHOD OF INSTRUCTION AND ABILITY LEVEL 



63 



SR 



I AFQT Mental 
Percentile Category Mean 



CMI 



II (SP + CMI) 



CI (GP) 



SD 



N Mean 



SD N Mean 



SD N Mean 



SD N 



93-99 
65-92 
49-64 
31-48 
21-30 
10-20 



1 $1719.22 . 750.08 152 $1598.55 345.44 6 $1712.74 738.31 158 $2929.97 1411.37 45 

2 2308.65 969.35 508 1818.22 616.64 167 2187.31 919.41 675 2650.93 1663.19 439 
3U 2929.55 975.91 390 2205.32' 668.05 310 2520.24 968.69 700 1950.22 1395.96 851 
3L 3226.99 1295,01 178 2293.60 717.23 281 2655.57 1081.62 459 1787.31 1012.63 341 
4U 3149.95 1035.02 10 2580.32 866.52 21 2764.07 946.16 31 3594.95 621.52 3 



4L 



N/A 



N/A 



N/A 



4704.67 
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FABLE 33. 



Task 


Percent 


Mean 


SD 


N 


Fact 


1-15 
16-30 


$2995.44. 
4196.78^ 


575;07 
495.43 


896 
66 


Category 


1-15 ' 
16-20 


2360.07 
1535.53 


665.94 
214.71 


2259 
957 


Procedure 


1-50 
51-100 


3086.84 
2010.59 


471.40 
872.19 


950 
2842 


Rule 


1-50 
51-100 


3093.06 
2755.93 


651.84 
478.72 


1793 
295 


Principle 


1-50 
51-100 


2761.42 
2958.58 


519.50 
495.42 


1124 
349 



TABLE 34. TRAINING COSTS BY TRAINING TASK AND METHOD OF INSTRUCTION 







II (SP + CMI) 






CI £GP) 




Task 


Percent 


Mean 


SD 


N 


Mean 




N 


Fact 


1-15 


$2488.37 


660.89 


454 


$3516.28 


* 471.64 


442 




16-30 


N/A 


N/A 


N/A 


4196.78 


495.43 


66 


Category 


1-15 


2484.52 


487.06 


1197 


2226.65 


615.82 


1062 




16-30 


1549.10 


282.22 


348 


1937.12 


164.11 


609 


Procedure 


i-50 


2958.62 


439.66 


312 


3149.54 


423.30 


638 




51-100 


2283.01 


717.72 


1773 


1558.74 


44.80 


1069 


Rule 


1-50 


2863.95 


810.68 


982 


3370.48 


378.65 


811 




51-100 


2755.93 


370.72 


295 


N/A 


N/A 


N/A 


Principle 


1-50 


2119.20 


656.14 


602 


3681.63 


502.08 


462 




51-100 


N/A 


N/A 


N/A 


2958.60 


495.42 


349 
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T1we to Complete the Course, In general, individual training costs were 
greater for graduates who took a longer time to complete their courses than 
for graduates who completed their courses in less time (R 2 increment = .38). 
But this was to be expected because training costs were computed using 
training time as a principal basis. 

Nonsignificant Results. The interactions of ability level by training task, 
and ability level by training task by method of instruction; main effects of 
sex of graduate and geographic location of school did not significantly 
predict training costs. 

SUMMARY OF FINDINGS 

s 

Summaries of the findings of the study are presented next in three 
tables. • 

The first table (table 35) lists the predictor variables assessed 
against four criterion variables: training time, training costs, end-of- 
course grades, and overall TAS ratings. Significant and nonsignificant 
results, are indicated in the cells. The "direction" of significant results 
(e.g., whether II has "better" effects than CI) is too complex for simple 
summarization. Consequently, directionality must be read from the relevant 
portions of the preceding text. 

Table 36 summarizes results of predictor comparisons by five generic 
task-specific TAS ratings used as criterion variables. Significant and 
nonsignificant comparisons/results are indicated. Again, however, the 
direction of effect must be obtained from the text. 

The final table (table 37) summarizes the salient findings of the study 
with respect to the effectiveness and efficiency of training as a function 
of II versus CI. 
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TABLE 35. SUMMARY TABLE OF STATISTICAL RESULTS FROM REGRESSION ANALYSES OF 
FOUR CRITERION VARIABLES BY PREDICTOR AND CRITERION VARIABLES 



* 




Criterion 


Variables 




1 « 


i ra i ni ny 


Tv» a *i fi t fin 

i ra i n i ny 


FnH_nf _ 
mia-ui - 


Uvcra 1 1 


rrea 1 c vors 


Time 


UOS tS 


uourse 


1 no 








UidQcb 


t\a u i ny b 


Method of Instruction*. (MOI) : 










II vs CI 


X 


X * 


X 


X 


SP vs CMI 


X 


.. x 


X 


- 


Ability (AFQT (A)) 


X 


'■ X 


X 


- 


MOI by A 


X 


X 


X 


- 


Percentage of Training Task (T): 










Fact- 


X 


X 


X 


X 


Category 


X 


X 


X 


X 


Procedure 


X 


X 


X 




Rule 


X 


X 


X 




Principle 


X 


X 


X 




T by A 










T by MOI 


X 


X 


X 




T by MOI by A 










Sex of Graduate 










Geographic School Location 










Training Time 


N/A 


X 







= Nonsignificant results 
X = Significant results 

N/A = Not applicable to analysis of. specific criterion 
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TABLE 36. SUMMARY TABLE OF STATISTICAL RESULTS FROM REGRESSION ANALYSES OF 
TASK-SPECIFIC TAS RATINGS BY PREDICTOR AND CRITERION VARIABLES 



Criterion 
(Task -Specific TAS Ratings) 


Predictors Fact 


Category Procedure Rule Principle 


Method of Instruction (MOI): 




II vs CI 
SP vs CMI 


X 


Ability (AFQT (A)) - **- 
MOI by A 

Percentage Training Task (T) 


X 

X 


T by A 

T by MOI 

T by MOI by A . 


X X * - 


- e - Nonsignificant results " - 
X = Significant results 
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TABLE 37. SUMMARY OF SALIENT FINDINGS OF PERFORMANCE MEASURES OF II 
GRADUATES COMPARED TO CI GRADUATES 



Outcome of II Graduates 
(Compared to CI Graduates) 



Measure 



Lower 



Same 



Higher 



Comments 



Effectiveness 






TAS Ratings 


X 


Although statistically higher for II graduates, 
overall TAS ratings support the generalization 
that II and CI graduates were considered by * 
their fleet supervisors to be adequately trained. 

Task-specific TAS ratings suggest that CI courses 
may be more effective in training rule and x 
principle tasks, whereas for II courses, procedure 
tasks. / 


End-of-Course Grades 


X 


Higher ability 11 graduates received higher 
grades than lower ability II graduates. 
Under CI, grados for different ability groups 
did not differ. 

Grades for lower ability groups under II and CI 
did not differ. 


Efficiency 






Training Costs 


X 


It cannot be concluded II is inherently more costly 
than CI since different courses with different 
contents and lengths were taught under each method 
and the Same cost factors were not necessarily 
included in the accounting base for all concerned. 


Training Time X < 




Because of tl inferences in the courses studied, it 
cannot be concluded that II necessarily reduces 
training time over CI. 

Higher ability II graduates completed their courses 
in less time than lower ability II graduates. 
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o 
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SECTION IV 
DISCUSSION 

The' results of the study are discussed in- this section. Brief discus- 
sions of the overall efficiency and effectiveness of II compared to CI are 
presented first. Discussions of the interrelationships of II with trainee 
ability levels and with the different types of generic training tasks aire 
then presented. • 

OVERALL EFFICIENCY 

Based on the measures used in this study, no one method of instruction 
was found clearly to be more efficient than another. Training efficiency- 
measured in temis of training time and training costs— was independent of 
fleet measures of training effectiveness. Student completion time and cost 
to produce a graduate were not reliable indicators of the level of training 
adequacy reported in the fleet setting for either II or CI graduates. 

At the school level, the finding that training costs were higher when 
producing graduates in less time under IT than under CI may not be due pri- 
marily to the method of instruction. Rather, it may be due to differences 
-in the factors comprising the cost data bases for specific courses. An 
analysis of the subcomponents of training costs of II and CI courses would 
appear to be an appropriate follow-up study into the reasons why training 
costs are different overall. Similarly, training time differences could be 
approached in terms of what aspects of courses under II or CI contribute 
significantly to student time savings. Further investigations of the 
efficient aspects of conventional instruction, self-paced instruction with- 
out computer support, and self-paced with computer support may even lead to 
an acceptable mixture of methods for use within a given course. 

OVERALL EFFECTIVENESS 

Fleet supervisor ratings of school training adequacy were used as the 
external criterion of training effectiveness. On this Criterion, the II 
group was rated significantly higher than the CI group. \Although .statis- 
tically significant, the mean values of ratings for both groups of graduates 
fell within the same range of values— a range that would bk considered to 
reflect satisfactory school training. Thus, the inference ean be made that 
the two basic methods of instruction are equally effective irNpreparing 
graduates for operational jobs. Within the II group, graduates\f CMI course 
were rated similarly to graduates of SP courses. Accordingly, th^two forms 
of 41, SP and CMI, can be considered equally effective from the data of this 
study. 

For the internal criterion of training effectiveness— end-of-course 
grades— the study results showed significantly higher average end-of-course 
grades to be associated overall with II. However, this result must be inter- 
preted in terms of the interaction between end-of-course grades and student 
ability levels. This interaction is discussed in greater detail below. 
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ABILITY LEVEL OF GRADUATES AND INDIVIDUALIZED INSTRUCTION 

The discussions below concern hc;v graduates of different ability levels 
were affected by II and cover both school and fleet setting criteria. 

SCHOOL LEVEL* At the school level, II benefited higher ability graduates 
more than lower ability graduates. The higher ability II graduates finished 
their courses sooner and, consequently, were less* costly to train than lower 
ability II graduates. The higher ability II graduates also achieved 
significantly higher end-of-course grades. This finding is consistent with 
previous research which has shown positive relationships' between student 
ability and school achievement (see Cronbach & Snow, 1977). 

It is interesting to note that those II courses providing end-of-course 
grades to students used normative (based on a group) measures of performance 
rather than an all-or-none (go/no-go) mastery criterion which is typically 
used in an II format. The use of this normative criterion in determining 
student* readiness. to graduate apparently did not influence fleet 
supervisors' perceptions of training adequacy for these graduates. Fleet 
supervisor ratings were not related to end-of-course grades. 

No definitive conclusions concerning training time and training costs 
> could be drawn for the diffeVent ability groups trained under CI. For this 
group, the higher ability graduates tended to be concentrated in courses of 
greater length. Consequently, over all CI courses, higher training costs * 
were associated with higher ability CI graduates than with lower ability 
graduates. This reflects an apparent selection bias which may have operated 
to place higher ability students in longer courses. 

No significant relationships were found between end of course grades 
for the different ability levels trained under CI. End of course grades for 
the higher .ability CI groups were essentially the same as those for the 
lower ability CI groups. 

The failure, in this study, to find a significant relationship between 
ability levels and grades for CI graduates is interesting since previous 
research has documented a positive relationship between these two variables 
(Cronbach & Snow, 1977). Perhaps something is occurring within the CI con- 
text that obscures or restricts the potential achievement of higher ability 
personnel. One possibility is that the grading system used is not suffici- 
ently sensitive to discriminate among students of different abilities. 
Another is that the CI context contains elements that adversely affect moti- 
vation of higher ability students to excel much beyond the minimum school 
performance criterion. For example, the higher ability CI graduates may 
have been exposed to a level of training content designed for the majority 
(i.e., mid to lower ability level) Of CI graduates and this material may 
have posed a limited challenge to the more capable students. Additionally, 
the higher ability CI graduates 1 motivation may be affected by their inabil- 
ity to control the amount of time they will spend in the CI course. 

To recapitulate, higher ability II graduates mastered relatively more 
course content than lower ability II graduates and did so in significantly 
less time. For the CI graduates, however, both end-of-course grades and 
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time to complete training were similar for all ability groups. These find- 
ings, taken together, suggest that a CI course converted to an II format would 
provide a training context in which the school performance of different ability 
level students is differentiated. For the newly converted course, training 
costs and training time would be predicted to be lower, and grades higher, 
for higher ability students. The average course completion time and the 
amount of improvement, however, can not be specified from the data of the 
present study. An issue for decision makers would be to weigh the costs and 
benefits of sending a highly able, trained graduate to the fleet sooner, 
against the effort involved in converting CI courses to II courses. 

There were no definitive findings in the present study to suggest whether 
lower ability students are more effectively and/or efficiently trained overall 
in II or CI courses. One point to consider here concerns the frequently 
made assertion that II provides a context in which lower ability students 
can achieve the same level of mastery of course content as higher ability 
students. The lower ability student, however, would presumably require greater 
time to reach this level. ' An underlying tenet of II is that students can 
proceed through training at their own pace. However, those responsible for 
training may have determined that necessary additional time for course mastery 
could not be permitted within current operational constraints. Hence, a 
tradeoff to be made concerns whether the probable increase in grades for 
lower ability students is worth the additional cost (i.e., greater training 
time) that would be required to bring all students to an equal level of mastery. 

FLEET SETTING. School performance measures, both time to complete training 
and end-of -course grades, were not related to fleet supervisor ratings of 
overall training adequacy of II and CI graduates on the job. Also, general 
ability measures were not related to fleet supervisor ratings. These findings 
may reflect those of previous research which has reported on the difficulty 
in 'demonstrating relationships among graduate ability, school performance 
measures, and measures reflecting job performance. The failure to find rela- 
tionships has been particularly noted when supervisor ratings, rather than 
job knowledge or. job sample tests, are used as indices of job performance 
(e.g., Vineberg, Sticht, Taylor, & Caylor, 1971; Wevrick, 1981). Supervisor 
ratings provide a greater opportunity for the occurrence of bias than do the 
other measures. Thus, rater bias is one possible explanation for the failure 
to find significant relationships among the variables. Another is that the 
instruments and procedures used for obtaining supervisor ratings are not 
sufficiently precise or sensitive enough to detect whatever differences may 
exist. 

-~ Evidence was obtained in the present study, however, that student learn- 
ing of different types of instructional content may be affected by the method 
of instruction used. Differences were observed on the fleet measures when 
school training content and items rated by fleet supervisors were classified 
by generic tasks trained in school. These findings are discussed below. 

GENERIC TRAINING TASKS AND INDIVIDUALIZED INSTRUCTION 

The discussion below concerns observed relationships between generic 
training tasks and graduate performance at the school level and in the fleet 
setting. 
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SCHOOL LEVEL. School performance of both II/ and CI graduates was differen- 
tially related to the amount of different types of generic tasks taught in 
school. Across the II and CI courses, graduates of those courses that taught 
larger amounts of category, procedure, or rul3 tasks cost less to train, 
finished their courses sooner, and received higher grades than graduates of 
courses featuring smaller amounts of these tasks. These same outcomes (i.e., 
lower cost, less time, and higher grades) were also observed for those II 
and CI courses that taught smaller amounts of fact or principle tasks compared 
to courses that taught larger amounts of these tasks. Fleet jobs for which 
students were required to learn relatively more fact or principle tasks in 
school than category, procedure, or rule tasks required longer training times. 
In addition, students achieved less (i.e., got lower course grades) in those 
courses. Conversely, courses in which training featured greater amounts of 
category, procedure or rule tasks tended to be shorter and student achieve- , 
ment was higher. 

The overall (i.e., across all courses examined) relationships found for 
generic training tasks were not differentiated by method of instruction or 
by trainee ability level. Directional changes in school performance measures 
as a joint function of generic task type and instructional method were incon- 
sistent. In addition, no evidence was obtained in the present study to suggest 
that school performance of different ability level graduates was affected in 
any systematic manner by different amounts of generic training tasks taught 
in the schools sampled. 

FLEET SETTING. For the fleet setting, significant differences were observed 
in overall training adequacy ratings for fact or category tasks. Graduates 
taught smaller amounts of fact or category tasks in II and CI courses received 
higher fleet supervisor ratings for overall training adequacy than graduates 
taught larger amounts of these tasks. These findings are similar to those 
found for fact tasks at the school level. However, they are opposite in 
diretion to the relationship observed for category tasks at the school level. 
These findings suggest that smaller amounts of fact tasks are easier to train 
and apply on the job than larger amounts of fact tasks. The difference between 
'school and fleet performance findings for category tasks indicates a lack of 
consistency about which no further interpretation was made. 

When the TAS items rated by fleet supervisors were classified by generic 
tasks trained in both II and CI courses, l\ was found to be more effective 
than CI in courses that taught primarily procedure tasks. Conventional 
instruction was more effective than II in courses that taught primarily rule 
or principle tasks. These results suggest that training tasks concerned 
primarily with generalized or cause-effect relationships (rules or principles) 
may be more adequately presented or learned under a CI format than under an 
II format. Training tasks concerned mainly with situation-specific, sequential 
"steps (procedures), on the other hand, appear to be more adequately trained 
under an II format than under a CI format. This suggestion implies that any 
one course could have both an II format and a CI format to present effectively 
different kinds of training content to students. 

In summary, the results concerned with the interactive effects of ability 
level and type training tasks with methods of instruction suggest that no 
one method of instruction is more or less effective/efficient overall. Indeed, 
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some combination of instructional formats within a given course may promote 
training effectiveness/efficiency to a greater extent than a single format. 
This combination of formats could be based on different ability levels of 
incoming students and/or different kinds of generic tasks taught in -the 
school. 

Table 38 presents a hypothetical version of how II and CI formats may 
be assigned to teach different amounts of generic tasks in a course consider- 
ing the ability level of students. The assignment of ■ II and CI formats is 
based on the findings in the present study, past research reviewed, and specu- 
lation. The purpose of presenting this table is to suggest one approach 
that instructional designers may use to address the issue of different formats 
within one course. 



TABLE 38. INSTRUCTIONAL FORMATS HYPOTHESIZED FOR DIFFERENT ABILITY LEVELS 
OF STUDENTS AND GENERIC TRAINING TASKS 







Training Task and Amount Taught* 






Fact 


Category . Procedure 


Rule 


Principle 


> 


H L 


H L H L ' 


H L 


H L 


High Ability 
. (mental categories 
1 - 3U) 


I** I 


II II 


C*** I 


C C 


Low Ability 
(mental categories 
3L - 4L) 


C C 


I I I C 


C C 


C C 



*H = Relatively high amount; L = Relatively low amount 
**I = Individualized format 
***C = Conventional format 



Note: High and low percentages for tasks taught are: 
Fact and category: H: 16-30%, L: 1-15% 
procedure, rule, and principle: H: 51-100%; L: 1-50% 



In table 38, under fact, it was assumed that higher ability students 
could impose their own structure on learning facts, but that lower ability 
students would require instructor support in learning large numbers of facts. 
For both higher and lower ability students, -an II format was chosen under 
category tasks because it was assumed that sufficent structure was inherent 
in the presentation of these tasks in a self-paced mode for all students to 
learn. An II format was chosen for higher and lower ability students under 
high amounts of procedure tasks taught, and was based on the task-specific 
TAS findings in the present study. A conventional format was assigned to 
lower ability students under low amounts of procedure task, based on the 
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task-specif ic TAS findings. Higher ability students were assigned an II 
format for low amounts of procedure tasks based on the assumption that suffi- 
cient task structure was inherent in the presentatioa of this amount of proce- 
dure tasks in a self-paced mode. A conventional format was assigned to all 
students under high amounts of rule tasks based on the task-specific TAS 
findings. Low amounts of rule tasks were assigned an II format for higher 
ability students based on task-specific TAS findings. A CI format was assigned 
for lower ability students based on the assumption that lower ability students 
would require instructor support in learning the generalization aspect of 
rule tasks. Conventional formats were assigned to higher and lower amounts 
of principle tasks trained to all students based on the task-specific TAS 
findings. 

The differences between II and CI formats need not be sharply divided 
in one course for present ing different tasks. A conventional format could 
still use an individualized environment (e.g., computer terminal, SP carrel) tt 
but couild provide instruction in small groups via peers or instructors. Recent 
evidence (Hopmeier, 1981) has suggested that screening incoming students for 
personality types can facilitate performance in a computer assisted instruc- 
tion (CAI) environment. Modifying an II context to incorporate CI format 
changes reduced student attrition and decreased training time for Navy enlisted 
personnel. These changes resulted from classifying an individual as intro- 
verted (does well in a typical II format) or extroverted (requires the "people" 
aspect of a CI format). In addition, as noted in Vineberg et al. (1971), a 
significant number of lower mental category military personnel depended more 
on the auditory mode rather than the visual mode (i.e., printed material) to 
obtain job-related information than higher mental category personnel. An 
issue raised by these findings, then, is whether lower ability students may 
be more extroverted when learning, and/or whether higher ability students 
tend to be more introverted in their preference. 
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SECTION V 

CONCLUSIONS AND RECOMMENDATIONS 

This section contains conclusions about the relative effectiveness/ 
efficiency of II for different kinds of training tasks and ability levels of 
trainees. Recommendations for II in Navy technical training are also 
presented. , 

CONCLUSIONS 

Results presented in this report support the following conclusions for 
the courses studied: 

1. Individualized instruction (whether it is self-paced 1 or computer 
managed) is just as effective as conventional instruction in preparing 
graduates for operational jobs. 

2. Individualized instruction benefits higher ability students during 
training more than lower ability students. 

3. Conventional instruction does not benefit one ability group more 
than another during training. 

4. Fleet performance ratings do not reflect school performance differ- 
ences or different ability levels of graduates. 

5. When course content is classified into generic training tasks, 
fact or principle tasks are more difficult and time consuming for students * 
in both II and CI courses than category, procedure, or rule tasks. Student 
end-of -course grades are also lower in courses teaching relatively larger 
amounts of fact or principle tasks. [ 

6. When the TAS items rated by fleet supervisors were classified by 
generic tasks trained in both II and CI courses, II was found to be more 
effective than CI when courses teach primarily procedure tasks. Conventional 
instruction was more effective than II when courses teach primarily rule or 
principle tasks. 

7. No one method of instruction is more or less effective/efficient 

in training different types of training tasks to students of different ability 
levels. Indeed, some combination of instructional formats within a given 
course may facilitate training effectiveness/efficiency to a greater extent 
than a single format. 

8. Definitive conclusions concerning the efficiency (training costs 
and training time) of II for Navy training could not be reached from the i 
data of this study. 1 

9. Geographic location of the school and sex of graduate do not signi- 
ficantly influence overall school or fleet performance, i 
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RECOMMENDATIONS 

Since this report is based on data analyzed across schools, no specific 
recommendations are offered to improve individualized instruction in a specific 
school. The following recommendations pertain to schools grouped by instruc- 
tional method employed, general ability rankings of students, or generic 
tasks taught. 

Based on these perspectives, the following recommendations are offered: 

1. Careful consideration should be given to any attempt to individu- 
alize indiscriminately in all schools in the Navy. Rather, the feasibility 
of employing different instructional formats (instructor involvement, self- 
pacing, adaptive sequencing, computer support) with differently weighted 
contributions in a given course should be investigated. A pilot implementa- 
tion and an ongoing evaluation of a chosen alternative could then be initiated. 
•The goal of such an investigation would be to select cost-effective training 
systems for different kinds of Navy schools. 

2. .An economic analysis of subcomponent categories of training costs 
between schools could reveal resource sources contributing different amounts 
of money to different aspects of training. Life cycle costs, as one indicator 
of long-term training efficiency, could be used in conjunction with training 
effectiveness measures to decide among training efficiency/effectiveness of 
alternative systems. ' 

3. Attrition of first-term enlisted personnel is an issue of current 
interest. Completion of first tour of duty could be used as a measure of 
attrition and could be collected on the graduates in the sample selected for 
this study. Analysis of attrition data would provide information on the 
relative influence of the technical schools, general ability level of the 
graduate, method of instruction, and/or training tasks on explaining potential 
differences in attrition rates among groups of graduates in the sample inves- 
tigated. Training effectiveness/efficiency measures at the school level and 

in the fleet setting could also be treated as potential indicators of attrition. 

4. More objective measures of training effectiveness need to be iden- 
tified and collected for definitive evaluations of alternative training systems. 
Supervisor ratings are subjective measures which may or may not adequately 
discriminate individual capabilities to perform on a job for which training 

was given. Consideration should be given to development and use of more 
objective measures such as job knowledge and job sample tests for determining 
training effectiveness. The relationships between supervisor ratings and 
these types of tests should also be established. 
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SECTION VI 
POSTNOTE 

As a result of the study, needs were generated for additional informa- 
tion about II. Consequently, follow-on efforts were initiated. One effort 
is concerned with SP training time estimates. Another is examining practices 
used in conducting SP instruction. These efforts were initiated in part by 
the findings of Freda, Ford, & Hall (1981) that indicated three different ■ 
data bases for student completion times in individualized courses and instruc- 
tor preference for conventional instruction over individualized instruction. 

SP TRAINING TIME ESTIMATES 

While the research literature indicates that II can be more efficient 
than CI, it is difficult to demonstrate this empirically within the context 
of a "business as usual" military training environment. One problem is that 
there are different sources of information concerning student completion 
times in Navy self-paced courses. Currently, the TAEG is examining several 
sources of information to determine which source provides the most reliable 
and accurate baseline estimate of time-to-complete training under SP instruc- 
tion. This information is indispensable to determining the training effici- 
ency of SP instruction. 

PRACTICE OF SP INSTRUCTION 

In conducting the present study, the realization arose that, in practice 
SP instruction is no,t a unitary concept. The term apparently has different 
meanings to different individuals. It was observed during visits to II schools 
that there are variations in the ways in which SP instruction occurs. These 
variations may affect what II can achieve. Thus, TAE6 initiated an effort 
to identify the variations in ; SP instruction and to assess both positive and 
' negative factors associated with the different self-pacing strategies as 
they affect training efficiency or effectiveness. The thrust here is not to 
demonstrate that SP instruction is an efficient training method (since this 
seems to have been amply demonstrated by previous research) but rather how 
best to achieve the efficiencies. Thus, this study, just underway, will 
develop recommendations for optimizing the conduct of SP instruction. 
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APPENDIX A 

SWWARY OF FINDINGS ON CAI AND CMI COMPARED TO CONVENTIONAL INSTRUCTION 

This appendix presents the Orlansky and String (1979) surmiary of 
.stud!es on the use of II in military settings. nummary or 
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Table a-i . summary of findings on cai and cmi compared- 
to conventional instruction 





Finding 

(Compared to Conventional Instruction) 




Measure 


CAI 




CMI 


Comments u 


Student Achievement 


Same or more 




Same 


Performance measured only at school. 
Relation between performance at school 

and on the job not demonstrated. 
Observed differences not of practical 
- importance. 


Course 

Completion 

Time 

A 


No. of 
Comparisons 


40 


8 *> 


'CMI: Most time savings maintained 

or increased with extended use. 


Time saved 
(Median) 


29% 


4455 






Range 31-39% 


12-68% 






No. of 
Comparisons 

0 


5 


7 


Computer support saves little time beyond 
that of individualized instruction 
(i.e., self-paced). 


. B 


Time saved: 
Individual- 
ized in- 
struction 
CAI 


64% 
69% 


51% 
CMI 51% 




Student Attrition 


About the same 




Slight increase 
may occur 


CAI: very limited data 

i»ni . pubbiuic aec 1 1 ne in student quality 


Student Attitudes 


Favorable 




Favorable 




Instructor Attitudes 


Unfavorable 




Unfavorable 




Cost 


Less due to 
student time 
savings 




Less due to 
student time 
savings 


Data limited and incomplete 


Cost-effectiveness 








Not known because cost data are limited 
and incomplete 



SOURCE: Orlanslcy and Spring (1979) 

A. Conventional courses converted directly to CAI or CMI. 

B. Conventional courses converted to self-paced first and then to CAI or CMI. 
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APPENDIX B 

ASVAB INFORMATION 

This appendix describes the subtests comprising the Armed Services 
Vocational Aptitude Battery (ASVAB). It also provides the routines for 
converting ASVAB scores to Armed Forces Qualification Test (AFQT) 
percentiles. The AFQT percentiles we*e used as measures of general ability 
for the present TAEG study. The information presented in this appendix 
pertains to ASVAB usage prior to October 1980. The TAEG study sample of 
graduates was administered the pre-October 1980 ASVAB version. After 
October 1980, several changes were made uo the item content and test score 
interpretation of the ASVAB.,, 
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TABLE B-l. ASVAB SUBTESTS 



Subtest 


Abbrevia 
tion 


uescripcion 


Seneral Information 


GI 


A 15-item' general knowledge test, primarily on 
sports, outdoor activities, automobile mechan- 
ics, and history. Testing time is 7 minutes. 


burner icail Operations 


' NO 


A 50-item speeded mathematical test, requiring 
elementary addition, subtraction, multiplica- 
tion, and division—3 minutes. 


Attention to Detail 
* 


AD 


A 30-item speeded test in which the examinee 
counts the number of Cs embedded in lines of 
Os— 5 minutes. 


Word Knowledge 


WK 


A 30 -item vocabulary test— 10 minutes. 


Arithmetic Reasoning 


AR 


,A 20-item arithmetic test requiring examinees 
to solve word problems— 20 minutes. 


Space Perception 


SP 


A 20-item pictorial test. 'Requires examinee 
to select the three-dimensional figure that 
could be made" from a flat pattern— 12 minutes. 


Mathematics Knowledge 


MK 


A 20-item test requiring knowledge of algebra, 
geometry, fractions, decimals, and exponents— 
20 minutes. 


Electronics Information 


EI 


A 30-item test requiring knowledge of electrical 
and electronic components, principles, and 
symbols — 15 minutes 


necnamca I comprehension 


MC 


A 20-item test about drawings illustrating 
mechanical principles— 15 minutes. 


General Science 


GS 


r» i lcih LCii iHcaburing KnowieoQe in tns 
physical (N = 10) and biological (N = 10) 
sciences— 8 minutes. 


Shop Information 


SI 


A 20-item test on examinee's knowledge about 
the use of shop tools and practices— 8 minutes. 


Automotive Information 


AI 


A 20-; tern test on automobile Darts", operations, 
or malfunctions— 10 minutes. 



1. Scores reported as Navy Standard Scores (NSS) having a mean of 50 and a 
standard devotion of 10 for an unrestricted recruit population. 

2. AFQT is computed by adding scores for .WK, AR and SP. The total is then 
converted to an AFQT percentile score. \ 1 
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TABLE B-2. PRE -OCTOBER 1980 AFQT CONVERSION: ASVAB FORMS 6 AND 7 



2S,.u?L 1 L5°? pUted by ad( ^ ing three raw com P° n ent scores: WK+AR+SP. The 



Total 


AFDT 


Raw 


r cr Ucil L 


Score 




70 


QQ 

yy 


69 


yo 


68 


Q7 
y / 


67 


Qfi 

yo 


66 


yo 


65 


94 


64 


93 


63 


y l 


62 


QQ 

oy 


61 

u J. 


ob 


fin 


83 




OA 

80 




11 


57 


/b 


•J u 


70 

16 


55 


71 


'54 


69 


53 


C7 
0/ 


52 


65 


51 


64 


50 


. 62 


49 


60 


48 


58 


47 


56 


46 


55 


45 


54 


44 


52 


43 


50 


42 


49 


41 


48 



Total 
Raw 
Score 



AFQT 

Percentile 
Score 



ASVAB AFQT Scores 

93 to 99 
65 to 92 
49 to 64 
31 to. 48 
21 to 30. 
10\to 20 
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40 


47 


39 


46 


38 


45 


37 


43 


36 


41 


35 


39 


34 


37 


33 


35 - 


32 


33 


31 


31 


30 


28 


29 


25 


28 


V 1 "' " •'num required 




for enlistment) 


27 


19 


26 


16 


25 


13 


24 


11 


23 


10 


22 


8 


21 


7 


20 


6 \ 


18-19 


5 ' 


16-17 


4 


14-15 


3 


12-13 


2 


0-11 


1 


Mental 


Category 



Upper 
Lower 
Upper 
Lower 



1 
2 
3 
3 
4 
4 



(3U) 
(3L) 
(4U) 
(4L) 
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APPENDIX C 
SME TRAINING PACKAGE 

t 

This appendix is a copy of the package used for training school subject 
matter experts to classify course learning objectives into the Instructional 
Quality Inventory (IQI) system. The material is presented in 4 parts as 
follows: 

Part I. Classification Procedures for SMEs Based on a Modified Version 
of the IQI System 

Part II. Question Guide 

i 

Part III. Practice Items 
Part IV. Classification Guide 

Material contained in the package was adapted from Ellis, Wulfeck and 
Fredericks (1979). , 
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PART L CLASSIFICATION PROCEDURES FOR SMEs BASED 
ON A MODIFIED VERSION OF THE IQI SYSTEM 

THE INSTRUCTIONAL QUALITY INVENTORY (IQI) SYSTEM 

The IQI classification scheme is shown below. Classification of the TAS 
items is determined according to: 

1. What the student must do; that is, the TASK to be performed. 

2. The type of information the student must learn; that is, instruc- 
tional CONTENT. 

INSTRUCTIONAL CONTENT 



FACT 



CATEGORY 



REMEMBER 



. 'TASK 
* LEVEL 



USE 

UNAIDED 



Recall or Remember the 

recogni/a characteristics 

names, p&^ts, of each cate- 

dates, places, gory and the 

vocabulary guidelines for 

definitions, classification, 
etc. 



PROCEDURE 



RULE 



PRINCIPLE 



Remember the 
steps of the 
procedure. 




USE AIDED 



Classify or 
categorize 
objects, events, 
ideas, accord- 
ing to their 
characteristics, 
with no memory 
aid. 



Apply the 
steps of the 
procedure in 
a single sit- 
uation oc on 
a single 
piece of 
equipment, 
with no 
memory aid. 



Remember the 
formula or 
the steps of 5 
the rule. \ 



Given category 
characteris- 
tics amd guide- 
lines, cate- 
gorize objects, 
events, ideas, 
according to 
characteris- 
tics. 



Given steps of 
the procedure, 
apply the 
procedure in 
a single sit- 
uation, or on 
a single piece 
of equipment. 



Apply the 
formula or 
rule to a 
variety of 
problems or 
situations , 
with no 
memory, aid. 



Given the 
formula or 
rule steps, 
apply the 
formula or 
rule to a 
a variety of 
problems or 
situations. 



Remember the 
cause and 
effect rela- 
tionships or 
the state- 
ment of the 
principle. 



Use the 
principle 
to inter- 
pret or 
predict why 
or how 
things 
happened 
or wjll 
happen, 
with no 
memory aid. 



Given a state- 
ment of the \ 
principle, 
interpret j 
or predict ! 
why or how I 
things hap- I 
pened or 
will happen. 
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What is being proposed is to classify each TAS item into one of the 1 
cells in the above matrix. Inspect the matrix briefly. Note the five types 
of instructional content across the top, and the three types of task levels 
down the side. Each cell contains a general description of the performance 
tasK. in the following section the detailed procedures for using the IQI 
are discussed. 

CLASSIFICATION PROCEDURE 

In order to classify each TAS item, you will have to answer only 4 questions, 
There are two steps in the classification procedure. The first step determines! 
what the student must do to perform the task (task level). Th§ second step 1 
determines the type of information learned to perform the task '(content type) 
Each step is carried out by answering two questions. 



Determining the Task Level 



Question 1 

Does the TAS item require the student to 



Remember 



or 



Use 



information? 

The remember-use distinction can usually be made by looking at the action 
verb in the item. Typical action verbs are listed below. The ones on the left 
usually indicate remember tasks, while the ones on the right usually indicate 
use-level tasks. 



REMEMBER 
name 

state (from memory) 

lisjt (from memory) 

recall 

remember 

rel ate 

write (from memory) 
recognize 

explain (from memory) 
describe (from memory) 



USE 

apply operate 

classify 1 repair 

analyze adjust ' ; 

derive calibrate 

demonstrate removee 

discriminate replace 

evaluate assemble 

solve disassemble 

prove calculate 

sort 

explain (on the basis of other knowledge) 
maintain 
compute 
predict 
perform 
determine 



troubleshoot 

load 

unload 
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Question 2 

If the student uses information, is this use of information 



Aided 



or 



Unaided 



If the task level is USE, the next question is to determine whether an 
AID is given. This can be done by looking at the conditions which the 
TAS item is to be performed. Anything that replaces the need for MEMORY counts 
as an aid. If the student must perform the task with no memory aid, then the 
task level is Use-Unaided. 



AIDS include: 



< Normal tools, materials, etc. 
« 

Determining the Content Type 



a list of procedure steps from a tech. 
manual or MRC card, 

■i 

a formula for solving problems, 
a list or table or chart of characteristics, 
a statement of a principle. 
, are NOT aids. 



Question 3 

Does the item only require the student to remember/use information 
! in situations or on types of equipment which were 
! specifically covered or previously seen in training? 



Yes 



or 



No 



The most important requirement to consider^ is whether or not the student 
will have to deal with situations or types of Equipment not previously 
encountered during training. For example, if all major instances of infor- 
mation required to perform the TAS item were not presented during trainings 
then answier No. (Examples: (a) student has to classify/identify security 
codes for different messages, (b) student has to calculate a numerical value 
using a mathematical formula, (c) student has to troubleshoot a piece of equip- 
ment for a faulty module.) 0 If all major instances were presented during 
training, then answer Yes. (Examples: (a) student has to name the four parts 
of a specific piece of equipment, (b) student has to perform a routine mainte- 
nance procedure on a specific piece of equipment.) 
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Question 4a 

If you answered YES to question 3, is the largest percentage 
of information learned to perform the TAS item a ' 



FACT 



or 



PROCEDURE 



FACTS. Facts are what you think they are. They are simple associations 
between objects, events, names, parts, functions, locations, dates, etc. 
Facts don t have to come in pairs; there may be three or four or more pieces 
of information that go together. For example, a student might have to remember 
the name, location, and function of each of the parts in some piece of equip- 
ment. The task is always for the student to recall them or, given one part 
of the fact, to recall the other parts. Facts by definition must be presented 
during training. 



Some key phrases are: 
The student will 



give the symbol for each 

match each ... with its 

list the names of each 

recall the dates of 

recall the location and function of each 



given the 



associated with each 



PROCEDURES . A procedure is a sequence of steps, performed in order, on 
a single piece of equipment or in a sirigle situation. At the REMEMBER level, 
the: student must remember the steps in order. At the USE level, the student 
is given a piece of equipment or a situation, and must perform the steps. 
The job requirements for procedures involve single pieces of equipment or. 
single situations, and the student does not have} to "generalize" to new 
equipments or situaitons. In other words, everything the student needs to 
know is presented during training. 



>ome key phrases are: 

The student will perform the 



steps operating 
process performing 
procedure for maintaining 
sequence lighting off 

etc. 
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\ 



(single situation 
or 

piece of equipment) 



• 5 


STEPS 
OF 

PROCEDURE 






> 



(change in situation 
or 

piece of equipment) 



Example: 

"The student will field-strip an M-16 rifle." 



(M-16 rifle) 



STEPS OF 
PROCEDURE FOR 
FIELD-STRIPPING 
AN M-16 RIFLE 



(field-stripped rifle) 



Question 4b 

If you answered NO to question 3, is the largest percentage 
of information learned to perform the TAS item a 



CATEGORY 



or 



RULE 



or 



PROCEDURE 



CATEGORIES . The CATEGORY content type is used when the job requires that 
a iarge number of possible objects, events, etc., be classified into, or 
identified as a member of, one of a small number of particular categories. 
Instead of having to remember each object and its classification, the student 
is given characteristics for each category, which allow him to classify 
objects, etc. , he has not seen before. 



Some key phrases are: 

The student will demonstrate the 



characteristics 
features 

definition of each 
attributes 



type of . . . 
kind of . . . 
category of 
classification 
situation 



(object/event) 
(obj ect/ event) 
(object/event) 



(large or infinite 
number of possible 
objects or events) 



EVALUATE 
CHARACTERISTICS 



(category 1) 
(category 2) 



(category n) 
or 

(doesn't belong 
in any category) 



small 

number 

of 

possible 
categories 
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Example: 



\ 



"Given a series of sonar scope displays, the student will classify them 
according to type of target." f 



(sonar display 1) 
(sonar display 2) 



(infinite number 
of possible sonar 
displays) 



EVALUATE 
CHARACTERISTICS 
OF DISPLAY TO 

DETERMINE 
TARGET TYPE 



(surface ship) 
(motorboat) 
(submarine) 
(whale) 

(no target present) 



small 

number 

of 

possible 
types 



RULES. A rule, like a procedure; is a sequence of steps and decisions. 
However, rules can be applied in a variety of situations or on* a 1 variety. of 
equipments, whereas a procedure is applied only in a specific situation or on 
a specific piece of equipment. Instead of having to remember each problem or 
go through the steps on each object,, the student is taught a RULE which allows 
him to deal with problems, objects, and events he has not seen before. 

Formulas and mathematical .calculations always involve the use' of rules, 
unless the student has a calculator or computer that does it for him. 



So(T|e key phrases are: 

The student will demonstrate the 



formula 
rule 

law for 

process 

steps 



solving 

deriving 

proving 

calculating 

determining 

etc. 



(problem 1 ) 
(problem 2) 



(large or infinite 
number of possible 
problems) 



STEPS OF 
THE RULE 

or 
FORMULA 



(answer to problem 1 ) 
(answer, to problem 2) 



(large or infinite 
rjumber of possible 
answers) 



Example: 

"Given any two values of current, voltage, or resistance in a circuit, 
the student will use Ohm's Law to solve for the third value." 



C-7 or: 

KJ Kj 



(E=40.v., 1=27 ma., R=?) 
(R=80\meg., E120 v., I=?) 



.(Infinite number of possible 
Ohm's Law problems) 



CALCULATE 
ACCORDING TO 
OHM'S LAW *, 



(answer to R=?) 
(answer to I=?) 



(infinite number of 
possible answers) 



PRINCIPLES. Principles involve explanations of why/how things work, or do 
not work the way they do work, or predictions about "what would happen if 
Principles are based on cause-effect relationships, theoretical statements, 
statistical associations, or physical or scientific n laws." Instead of having 
to remember eacji possible situation or event and its effects, the student 'is 
given a PRINCIPLE which summarizes the "hotf" or "why" o£ general situations or 
which allows the student to predict what is likely to occur in a variety of 
situations. 



Some key phrases are: 
* The student will demonstrate 



the principle of 
the explanation of 
how ... 
Why ... 

the reasons for 



(situation requiring 
explanation or prediction) 



(other situation requiring 
explanation or prediction) 



CAUSES AND EFFECTS 
or 

THEORETICAL STATEMENTS 



(explanation » 
or prediction*) 



(explanation 
or prediction) 



Example: 



Based on his knowledge of electronic theory, the student will predict 
the effect .in the circuit shown below if theload resistance, or the 
filter capacitor, were shorted. 



1 



(situation requiring 
prediction - load 
resistance shorted) 



(filter capacitor 
shorted) 



^-ELECTRONIC THEORY - 



(prediction 
about resulting 
circuit behavior! 



(prediction 
about resulting 
circuit behavior 
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To/suramanze, there are five content types. Two content types involve 
performing jobs in situations or on equipment specifically encountered during 
c!H-l- 9, -M Three co " t 5 n * ^P« s involve dealing with equipment or situations not 
!P ec } f ]cally presented during tra ™™9- This explanation is presented in the 
to l lowing table: 



Content T.VDe 


All Major Instances 


Covered in Training? 


FACT 


Yes 


CATEGORY 


No 


PROCEDURE 


Yes 


RULE 


No 


PRINCIPLE 


No 



rATrpSr£ ime Vl aSslflcation can be confusing, especially between FACTS and 

"rImSJd Tu? n ^DH tW !u n . P ^ 0CEDURES and RULES - The wa * t0 resolve Problems is to 
REMEMBER THE JOB ; that is, to consider carefully what the student must be able 
to do after instruction. 

. Again, the most important thing to consider is whether the student will 
have to deal with objects or situations that he has not seen during training, 
hor example, if the student were required to sort or classify things according 

•♦J 5J£ characteristics, and if the student on the job were going to be dealing 
with things not seen during training, then the content would be a CATEGORY. 
However, suppose instead that there were only seven objects the student would 
ever see. Then, it would be more efficient to teach each object and its category 
name as a FACT (seven facts total). ^uryury 

Similarly, RULES are taught so that the student can apply his knowledge to 
situations he did not encounter in training. However, suppose the situations 
are so similar that "if you've seen one, you've seen them all." This would be 
more efficiently taught as a PROCEDURE. 

Example: FACT vs. CATEGORY 

"Given a variety of metal fasteners, the student will sort them according 
to type (bolts, screws, studs, or rivets)." 

This could be taught as a CATEGORY: the student could be taught the 
characteristics of bolts (fine threads, blunt end, etc.), the characteris- 
tics of screws (coarse threads, pointed end, etc.), the characteristics 
of studs /(no head, fine threads, etc.), and the characteristics of rivets 
(no threads, etc.). However, one bolt is pretty much, the same as any other 
bolt, and the same for screws, studs, and rivets, except that they come 
in different sizes. Therefore, it might be more efficient to teach these 
as four FACTS: bolt - appearance, screw - appearancej etc. The confusion 
here can be solved if the job requirements are determined. If there are 
lots of different metal fasteners, and the student will see new bolts, 
etc., on the job, then the content type is CATEGORY, ilf there are only a 
few, and they're all nearly alike, then the content type is FACT. 



Example: PROCEDURE vs. RULE 

"Given a word in print, correctly spelled, the student wilV look up the 
word in a dictionary, and state its definition orally." 



'This might appear to be a RULE: There are a large number/of possible 
words (inputs), and a large number of possible definition's (outputs) 
However, since the spelling is given, its easy to look u 
Find the first letter of- the word find that-chapter in 
find the second letter, find that section of the chapte 
most efficiently taught as a' PROCEDURE. 



the word: 
;he dictionary, 
etc. This is 



However, suppose the word was given orally and not sp( 
then be a fairly complicated RULE, involving listeninc 
translations, etc. 



lied. This would 
skills, phonemic 
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PART II. QUESTION GUIDE . . 

\ a„ Th ! f ? ]1 °w in 9 sequence of questions should be used to assure consistent 
judgments in classifying eachTAS item: ".«*ure consistent 

1. Does the TAS item require the student to remember or/use information? 

2. If USE information, is performance AIDED or UNAIDED? 

3... Is the situation/equipment in the item specifically covered during training? 
PROCEDURE? ' 1S ^ inStructional content in the it®" primarily a FACT or 

4b * RULEfor°PRjNcipLE? the inStrUCtional content in the ite * Primarily a CATEGORY, 
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PART III. PRACTICE ITEMS 



Use the preceding question guide to classify the following sample skill/ 
knowledge items. The items are classified according to the scheme presented 
earlier. Compare your classification with the one given for each item. 



Some of the items will be difficult to classify. There are three reasons 
for this. First, many of them are\not "good" ones; they are written .in such a 
way that it may not be clear what behavior is required or what content is to be 
taught. (They are, however, fairly\ typical.) Second, all of thenware taken 
out of context and may deal with unfamiliar topics. Therefore, they are diffi- 
cult to classify, because informatio^ about the job is not provided. Tiiird, some 
examples were chosen deliberately to be hard to classify, so that classification 
problems could be illustrated. \ 

For the reasons given above, the reader should not expect to be able to 
classify perfectly these objectives and test items (or any others) immediately. 
Since the most important step in classification is REMEMBER THE JOB, the reader 
should learn to ask the "right" questions of job experts, so that bad items can. 
be revised, and so that unfamiliar topic areaS can be classified reliably. The 
examples are intended to illustrate this "question asking" process. 



ITEM 
1. 



Tack Level? 



State the rule for finding total inductance in a series 
circuit. 

REMEMBER 



Content Type? 



Does the student have to remember something, or 
perform a task? In this case, the student simply has 
to recall the correct rule for finding total inductance. 
Therefore, the task level is Remember. 

RULE 



2. 



Task Level? 



The student must remember a rule for SOLVING for 
total inductance. " SOLVE" is a keyword for RULE. 
The formula for total Inductance involves a series of 
mathematical operations applicable in any series . 
circuit having inductors with various possible values. 
The process, .then, is a series of steps which apply 
across situations.* ' 

It 1s possible toconfuse a piece of information 
taught at the Remember level for Categories, Procedures, 
Rules, or Principles, with Remember-Fact information. 
The difference is that Facts cannot be used immediately . 
In this case, the student could use the rule to solve 
inductance problems. 

Given pictures of the collar devices for different 
ranks of Naval officers, identify the ranks they 
represent. 

REMEMBER 



Content Type? 



The task here is "I give you a picture of a collar 
device, and you tell me the rank." Because there is 
a limited number of different collar devices, the 
student can learn each device and its name. The task 
is Remember. 

FACT 



The content type is Fact, because the student has to 
memorize pairs of symbols and names . 

Suppose instead that there were a large number of 
different ranks and collar devices, but that different 
groups of them had similar characteristics. If the 
student's task were- to look at a collar device and 
identify which group it belonged to, then this objective 
would be Use-Unaided Category. 
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ITEM 
3. 



Task Level? 



Given any resistor with four color bands, state the ohmic 
value indicated by the color bands. 

/ 

USE-UNAIDED 



Content Type? 



If ther ; e were only a limited number of resistors, the 
student would ever have to deal with, he could\memorize 
each resistor 1 s colors and their values and the task 
level /would be Remember/ However, there are many 
different resistors with many different color patterns. 
Therefore, the student / must be given a scheme for 
determining ohmic value from the color pattern . When 
the/ student applies this scheme to any particular 
resistc, he is USING the scheme. If he has no memory 
aid,, then the task level is Use-Unaided. 

PROCEDURE 



Task Level? 



Content Type? 



In order to make the content type decision, we need 
to know more about the content than is given ^n the 
objective. In particular, we need to know what the 
scheme for determining ohmic value from the colors is. 
This is a good t/ime to consult a subject-matter expert. 
It turns out that the scheme is a fairly simple sequence 
of steps: the /first two color bands indicate the first 
twQ significant figures of the ohmic value; the third 
band is the number of zeros to add. (The fourth band ~ 
determines tolerance.,, not value.) If the scheme were 
more complicated, and involved complex calculations, 
the content type would be Rule. This one is simple 
enough to be a Procedures 

Thjs objective should not 'be confused with simply 
learning the meanings of the colors. Those are Facts 
which the/student .must remember. Those facts support 
this Use-level objective.. 

State steps involved in message reception and duplicate 
checking, as/ listed in the current edition of NTP-4. 

REMEMBER 



The student is asked to recall the steps* not to 
do anything with them. 



PROCEDURE 



"Steps" is a keyword for Procedure, and in this /case 
the task described in the test item appears to oe 
procedural. It would be a good idea; however, /to check 
with a subject matter expert to make sure. 
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Describe the -principles of operation of a rotary gear pump. 
Task Level? REMEMBER or -USE-UNAIDED 

It is difficult to determine whether this objective 
is Remember or Use-unaided without further information, 
If the objective had said "The student will recall 
•• . tne principles then it would clearly be 

Remember. On the other hand, if the objective had 
said "The student will use his knowledge of hvdraulic 
theory to explain how a rotary gear pump works," then, 
it would be Use-unaided. It is necessary to determine 
what the student must do after training, and write the 
„ objective appropriately. 



Content. Type? 



PRINCIPLE 



Task Level? 



This objective requires the student to explain how 
or why the pump works. The content type is Principle. 

Disassemble and reassemble the globe valve using your MRC 
job program card. The valve, ^ags, Prussian blue, gasket 
material, packing, and tools are on the workbench. 

USE-AIDED 



Content Type? 



The MRC job program card gives the steps to be 
followed, so the student does not have to remember 
them. I ■ - 

/ 

PROCEDURE / 



Task Level? 



This is a series 'of steps performed on a single 
piece of equipment. 

For each of the call signs listed below identify whether 
it is a Navy ship call sign, a Navy shore call sign, an 
indefinite call sign, a voice call sign, a task organization 
call sign, or/not a valid Navy calTsign. 

USE-UNAIDED 



Content Type? 



The task level of this test item depends on complexity 
of the task. If there were just a few call signs, then 
it would be easy to memorize their types, and the task 
would be Remember. Here, though, it turns out that 
there are several thousand call signs, and the student 
must use some classification scheme to identify their, 
type.- This test- item requires the student to use the . 
scheme with no memory aid. 

CATEGORY . 



In this case there is a large number of call signs 
.which can be classified into a small number of cate- 
gories or types. 
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8- Given the formula for Ohm's Law, and two values, solve for 

the third value. 



Task Level? 



USE-AIDED 



Content Type? 



The student must use Ohm's Law to solve for a value. 
In this case, he is given the formula as a memory aid, 
He does not have to remember it. The task level is 
Use-aided. 

RULE 



"Solve" is a keyword for Rules. When the student is 
asked to solve something, he must perform a series of 
mathematical operations which result in unique solutions 
depending on the values used. 



9. 



, Recall the duties of a gun captain. 

Task Level? REMEMBER 



Content Type? 



10. 



^Task Level? 



Content Type? 



"Recall" is a keyword for the Remember level. The 
student does not have to perform the duties, only 
remember what they are. 

FACT or PROCEDURE 



The content type is Procedure if the studentjs 
expected to remember a specific sequence of steps 
which the gun captain must perform. However, if the* 
student is to recall only the general functions of a 
gun captain, the objective would be a Fact. Again, it 
is necessary to consult a subject-matter expert to 
make thi^ decision. 



"File 30 drill messages representing Top Secret, Secret, 
s 
c 

USE-UNAIDED 



special category, readdressed ytgeneral messages, and other- 
classified and unclassified messages in the correct files." 



4 

The student must- perfonirthe task, with' no. memory aid . 
The task level is Use-Uftaided. 

CATEGORY or PROCEDURE * 



If the student must inspect each message! and determine 
category membership according .to its critical charac- 
teristics, then the content type is Category. If, 
however, the classification of the message is obvious 
(e.g. the classification is printed on the top line 
of the message), then this is a procedural task 
involving filing. Once again, the services of a 
subject-matter expert are required. 
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Task Level? 



Give the names of the components represented by the 
following schematic symbols... 

REMEMBER 



Content Type? 



In order to. answer this test item, the student would 
have had to memorize the names' of the schematic symbols; 
therefore, the task level is Remember. 

FACT 



Task Level? 



The key phrase in this test item is "Give the names." 
Simple associations between objects or symbols and 
their names are Facts. 

Given' the explanation of the principle of supply of a mili- 
tary force from the text, discuss how this principle applies 
in Hitler s attack on Russia, the Battle of Midway, the 
Battle of the Bulge, and Sherman's march through Georgia. 

USE-AIDED 



Content Type? 



The student is given an explanation of the principle; 
therefore the task is Aided. The task is Use because 
the student must apply the principle to explain specific 
battles. It is assumed that these battles are not 
discussed in the text and have not been described in 
class. If .they had been, the task level would be 
Remember and there would be no reason to give the 
student an Aid. 

PRINCIPLE 



Task Level? 



The student is given a principle and asked to interpret 
the outcome of specific battles. The key words here 
are "principle" and "discuss." • 

Recall 1n writing the lofargram characteristics for the 
following types of contacts: ( merchant ship, aircraft 
carrier, destroyer, whale diesel submarine, nuclear submarine. 

REMEMBER 



Content Type? 



The student does not have to do anything with the 
characteristics except recall them; therefore, the 
task level in Remember. 

CATEGORY or FACT 



The student is asked to memorize characteristics of 
several categories of contacts presumably because 
he/she will later have to look at lofargrams and 
determine what the contact type is Therefore, the 
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Task Level? 



content type is either Fact or. Category depending 
on whether or not this identification requires 
generalization. If the lofargrams within each cate- 
gory are pretty much identical, then the content type 
would be'Fact. If not, then the content type is 
Category. The guidance of a subject-matter expert 
is required. 

Solve for inductive reactance in a circuit, given frequency 
and. inductance. 

USE-UNAIDED 



Content Type? 



The task level is Use because the student is required 
to solve problems. It is not aided because he/she 
is not given the formula. 

RULE 



Task Level? 



The word "solve" is a keyword for Rule. Here the 
student must remember the formula for inductive 
reactance and then use it to solve problems. 

Perform the steps required to accept, verify, and log 
messages to be transmitted via teletype tape." 

USE-UNAIDED 



Content Type? 



The student must "perform the steps," so the task level 
is Use. The student has no memory aid, so the task 
is Unaided. 

PROCEDURE 



Task Level? 



Content Type? 



The word "steps" is a keyword for Procedure or Rule. 
If accepting, verifying, and logging is complicated 
enough so that doing one or two does not mean that the 
student could do any new message, then the content type 
would be Rule. In this case, though, the steps are the 
same for every message, so the content type is Procedure. 

State the principles of operation of a jet pump as described 
in Propulsion Engineering Mod 6, Lesson 2. 

REMEMBER 



The student simply has to recall how a jet pump works, 
as described in his training manual. 

PRINCIPLE 



The student is asked to remember "how" something works. 
Explanations of how things work are Principles. 
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Use the principle of electromagnetic induction to describe 
the operation of an AC generator. 

Task Level? USE-UNAIDED or REMEMBER 

The task level depends on what the student will be 
taught during the course; that is, on what the other 
objectives are. if the student must use his knowledge 
of electromagnetic induction to describe something not 
yet taught, therNthettask is Use-unaided. If the 
student had been taught the principles of operation 
of an AC generator on the basis qf electromagnetic 
induction, then all the student would have to do is 
Remember. 



Content Type? 



18. 



Task Level? 



Content Type? 



PRINCIPLE 



The student must explain "how" an- AC generator works. 

Given the guidelines for determining message security 
classification, determine the security classification (Top 
Secret, Secret, Confidential, or Unclassified) for outgoing 
messages. \ 

\ 

USE-AIDED 



The student must perform a task: determine appropriate 
security classifications for a variety of messages. 
The task level is Use. To the extent that the given 
guidelines provide sufficient information to determine 
classifications for varied messages, the task is Ajded. 

CATEGORY 



It is expected that the student could take any message 
and determine which of the four security classifications 
it belongs in. On the job, he will be required to apply 
the guidelines to messages he has not seen before. 
Presumably the messages will be varied enough so that 
doing one is not like doing them all. The given guide- 
lines should contain characteristics of messages that 
help determine the type of classification. 



From the examples above, it should be clear that the k=y to classification 
is REMEMBER THE JOB. Whenever there is doubt about the classification of an 
item, a subject-matter expert or technical manual should be consulted, so that 
information about the job can be obtained. 
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PART IV. CLASSIFICATION GUIDE 

Table C- 1 presents alphabetically some use-level action verbs classified 
by instructional content. The checkmarks represent typical classificaticns\of 
these verbs by information content. Note that these classifications could \ 
change due to: (1) the content of the skill item from which these verbs 
were taken or (2) relevant expert knowledge provided by a subject matter expert 
(SME). Thus, use this table only as a guide when classifying TAS items. 



\ 

0 
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TABLE C-l . ACTION VERB MATRIX 


USE-LEVEL 
ACTION VERBS 


INSTRUCTIONAL 
- CONTENT 




FACT CATEGORY PROCEDURE RULE 


PRINCIPLE 


ADJUST 


X 




Alll A i \/*yr • 

ANALYZE 




X 


APPLY 


X 




ASSEMBLE 


X 




CALCULATE 


X 




CALIBRATE 


X 




CATEGORIZE 


X 




CHOOSE 


X 




CLASSIFY 


X 




CUMrUFt 


X 




UcKlVt 


X 




DETERMINE 


X 




DlAbnOSE 




X 


DISASSEMBLE 


X 




A T c/*n T Li T ft! A Tr 

DISCRIMINATE 


X ' 




EXPLAIN 




X 


r*if a i t • ft Tr* 

EVALUATE * 




X 


IDENTIFY 


x - 




LIST 


. x 




1 Ann 

LOAD 


x 




MAINTAIN 


x 




MATCH 


X 




NAME 


X 




OPERATE 


X 




DCDCODM 
rcKrUKn 


X 




PREDICT 




X 


PREPARE 


X 




PROVE 


- X 




RECOGNIZE 


X 




REMOVE 


X 





TABLE C-l. ACTION VERB MATRIX (continued) 



* 


• 




INSTRUCTIONAL 
CONTENT 




USE-LEVEL 
ACTION VERBS 


FACT 


CATEGORY 


PROCEDURE RULE 


PRINCIPLE 


REPAIR 






X 




REPLACE 






X s 




SELECT 




X 






SOLVE 






X 




SORT 




X 






TROUBLESHOOT 








X 


UNLOAD 






X 





V 



>5 
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APPENDIX D~ 
COST DATA 



the n2lL? P ?r5 1X d " c r 1b f s how Gaining cost data were computed for use in 

o, %S 25 5f£««i? rt i e|iy for c 5 oos1ng among alternate * rainin 9 ***** 

on cost and effectiveness bases 'is also described. | 

rnctQ Tr a JlIlL 008 * 8 sh ? uld include both complete investment and operating 1 
'2I£ Inve ? tmen i co * ts are "front-end" expenditures associated with such 
cos™ are e "on a P ofnJ , .. ClaSS , r T buildi "9 s > curriculuum development. Opening 
Itlff 9 ° in9, repetitive expenditures. They include, items such as ' 

^^V^X^?"*- student costs < pay * health > ™. 

fcu a i Mlr h It£ ^S 1 ? ?°^ rse Costing Data Base maintained by the Chief of 

SJJmm 2 and Joining (CNET , many investment costs, such ; as 

b£e llFJlt CUrriC S! Um devel °P' nent costs, are not included The data - 

a?lowa^ces b ^l°SnH rating ^ W 5 1ch - are priniaril * militar * ™ a " d 
reflerts mi Sv 2?,?? era J in S J nd + maintena "ce, Navy (O&MN) costs. MPN 
oav SL c,,n! a - y Staf ! and s ? udent costs: >0&MN costs account for civilian 
r^«??f m S P ! eS * and * ravel - An "other" category is also used to 
] f y. ""scellaneous items. Based on the cost data available for the 

Scent SjtSS% Pr6Sent ™ E \^y> MPN accounted for approximate]? 74 

^pnl'n???/ 0r / P ^- Xlmately 26 perc?nt ' and " other " ftr less than 1 . 
percent of total training costs. 

i 

r*nol h r 1ncremer * al costing model (System 1)1 was used with the CNET Per 

Sal tSlSinn ^c" 9 * 5 3Se 1n the 5 resent TAEG stud * t0 calculate the 
fbased on Sy ?q Snn to P roduc f one graduate per average course 'session 
dollars). Costs to produce an individual graduate per 
SS,,;2 course " ere computed by multiplying the. average course cost to 
Zntl 6 9 raduate ' °y the ratio of a given student's time to complete the 
SSEIr? the . av 5 ra 9 e course completion time. This metric provided the 
m ?n ?hI eqUire J t0 USe ^ ra1n1ng costs as a criterion variable (see section 
III In ? He r ?° r l concernin 9 training costs for application of the metric in 
the analyses) However, since the full measure of resource and development 

™5 S LT n0t ade ?!!\ tely Stated in the data base > the results of KalSlS 
cost analyses should be viewed with caution. y 

X Loi * W0P u* ^thia^elvington, and J. M. Corey, . Incremental Costing 
Model for Use with the CNET Per C apita Course Costi ng Data Base : S V rL I 

Sanders ^ St^t ,rainin9 An ? and Eviauati ^ 

*|S^^^ Defense Management . 
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The selection of appropriate variables and measures is important not 
only for training costs but also for training effectiveness. Figure D-l 
presents a theoretical outcome matrix of possible decisions based on cost 
and effectiveness' factors underlying the alternative systems/programs being 
compared. Full measures of costs and effectiveness are required to diminish 
the occurrence of dilemmas in choosing a cost-effective alternative. 



EFFECTIVENESS 



COST 





LOWER 


SAME 


HIGHER 

» 


HIGHER 


NO 


NO 


? * 


SAME* 


NO 


EQUAL 


YES 


LOWER 




YES 


YES 



Figure ,0-1. Decision Outcome Matrix of 

Cost and Effectiveness Comparisons 



The Decision Outcome Matrix shown considers two factors: cost and 
effectiveness. Each factor is gradated ordinally. Each matrix- cell 
represents the joint outcome of a specified alternative system/program, being 
compared to another on cost and effectiveness factors. It is assumed that 
decision outcomes can be obtained from at least an ordinal scale of 
measurement. 

Note that out of the possible 9 outcomes, the two marked by a "?" 
suggest other areas of possible concern in cost and effectiveness 
comparisons. More factors may be taken into consideration when a highly 
effective system is the most costly alternative, or when a less effective 
system is the least costly. The joint outcome may be expressed as the 
desirability and/or certainty of selecting a specified alternative 
system/program. For example, system A costs less than, and is more 
effective than, system B. Thus, a desirable outcome of YES is indicated in 
the matrix for selecting A. . However, if A costs more, as well as being more 
effective than B, then a "?" ■ desirability is indicated. 

One strategy, to prepare for all possible outcomesi"is to use a multi- 
variable , multi -measurement approach to determine cost and effectiveness. 
The multi-variable aspect of this approach refers to an expansive effort to 
select those classes of measures that contribute totally or proportionately 
to the cost and/or to the effectiveness of the comparison systems /programs. 
Then, once the variables are selected, various measures can be employed to 
represent the same or different variables. This multi-measurement aspect 
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increases the probability of observing a correlated pattern among the 
measures for all the variables used to reflect cost and effectiveness. 
Further analyses are conducted to determine the significance of obtained 
correlated patterns. The multi-variable, multi -measurement approach may 
also lessen the occurrence of "?" outcomes, if such outcomes are 
predominately influenced by an insufficient number of variables and/or 
measures selected. 
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APPENDIX E 



COURSES INCLUDED 



IN THE STUDY 



~+..a The ^ u11 narnes and location s of the schools/courses examined durina the 
study program are presented in this appendix. 9 



11 
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TABLE E-l. COURSES INCLUDED IN THE STUDY 



/ 



School /Course Name 




Symbol 


Location 


Method of x 
Instruction 


iull Maintenance Technician 
"Class "A" School. Phase II 


/ 


HT-II 


San Diego, CA 


Mixed* 


Radioman "A" Sea School 


- 


RM-SEA 


San Diego, CA 


GP 2 


Radioman "A" Shore School 




RM-SHORE 


San Diego, CA 


GP 2 


Damage Control Assistant "A2" 


Course 


DCA 


Treasure Island, 


CA GP 3 


tall Maintenance Technician 
Class "A" School Phase. I 




HT-1 


Treasure Island, 


CA GP 


damage uontro i Assistant a^ 


Course 


DCA 


Philadelphia, PA 


SP 


Hull Maintenance Technician 
Class "A 11 School Phase I 




HT-1 


Philadelphia; PA 


GP 


Hull Maintenance "Technician 
Class "A" School Phase II 




HT-II • 


Philadelphia';: PA 


SP / " 


Electrician's Mate "k" School 




EM 


Great Lak.es,. I L 


' GP 


Engineman "A" School 


- - 


EN 


Great Lakes, IL 


4 

CMI 


: ire Control Technician 
"A" School Phase I J 




FT 


Great Lakes, IL 


GP 


gunner 1 s Mate "A" Schiol 




GM 


Great Lakes, IL 


GP 


Machinist's Mate 600 psi "A" , 


School 


MM-600 


Great Lakes, IL 


SP 5 ' 6 


Machinist's Mat/ 1200 psi 11 A 11 


School 


. MM-1200 


Great Lakes, IL 


SP 5 ' 6 
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TABLE E-l. COURSES INCLUDED IN THE STUDY (continued) 



School/Course Name 


Symbol 


/ 

Location 


Method of 
Instruction 


Instrumentman "A" School 

/ 


IM 




or 


/ 

Aviation Support Equipment 
Technician Electrical "Al" Course " 


ASE 


/Millinaton TN 


GP 


Aviation Machinist's Mate "A" School 


AD j 


Millinaton TN 


CMI^ 


Aviation Electrician's Mate "A" School 

• 


AW ••/ 


Millington, TN 


GP 


Aviation Technician Training 
Deviceman "A" School 


td/ 


Millington, TN 


SP . 


* 

Personnelman "A" School . . 


PN 


Meridian, MS 


SP 


Yeoman A" School 


/ YN 


Meridian, MS 


SP 


Disbursing Clerk "A" School / 


dk' 


Meridian, -MS 


X SP 


Aviation Storekeeper "A" School / 


AK 


Meridian, MS 


/* 

7 SP 


Ship * s Serviceman "A 11 School / 


SH j 


Meridian, MS 


G6 k ' . 



Group-assisted self -paced" (GRASP) 

> / i 

"CMI data from RM Basics obtained from CNTEC^TRA (CDP 6144; 894 records) 
^Officer course 

*CMI data from PE Basics obtained from CNTECHTRA (CDP 6261; 359 records) 
j / 1 \ 

CMI* data from PE Basics obtained from CNTECHTRA (CDP 6262; 552 records) 



"1978 student training records obtained from\ National Archives via CNTECNtRA 
and PE School. 

7 CMI data- obtained* from/cNTECHTRA. 



Technical Report 117 

\ 

APPENDIX F \ 

DISCUSSION OF THE PARTIAL HIERARCHICAL REGRESSION MODEL 

The partial hierarchical regression model used in data analyses is 
discussed in this appendixr Terms used in the model are explained and the 
model is specified in mathematical notation. 

DESCRIPTION OF THE PARTIAL HIERARCHICAL REGRESSION MODEL 

A partial hierarchical regression moclel was used to analyze the data of 
the study. A hieralrchial model is one in which the predictors are* entered 
cumulatively into the, regression equation according to a. specified hierarchy 
(or order) dictated in advance by the purpose and logic of the study. In 
the present TAEG study, this hierarchy was directed by the tasking 
questions; method of instruction was entered first;/ ability level and 
training- tasks second; and the two-way and three-way interactions formed by 
the previously entered predictors were entered subsequently. Since the „ 
predictors were entered as groups (sets) in each step of/ the equation, 
rather than entered individually, the model is called partial hierarchical 
regression. In partial hierarchical regression, the variables within each 
group (or set) are analyzed in a' forward stepwise manner 3 at each step of 
the equation* and each group of variables is analyzed hierarchically between 
sets. The for all predictor sets can thus be analyzed into increments in. 
th6 proportion of the criterion variance due to the addition of each new set 
of predictors to those higher in the hierarchy. These increments in are 
squared multiple semipartial correlation coefficients and represent the 
increments of criterion variance accounted for by a given set of predictors 
beyond what has already been accounted for by the^prior sets. An advantage 
of the hierarchical mojfel is that- once the order of the predictor sets has 
been specified, a unique partitioning of the tdtal criterion variance 
accounted-fbr by all predictor sets may be made. ; 

An issue raised by the use of the partial hierarchical 4 regression model 
concerns the basis for determining the extent of contribution of any group 
.of variables once th£ entry order is specified in-the equation. Since the 
contribution of the first set of predictors is determined from its initial 
entry in the equation, subsequent potential changes in the contribution by 
the first predictor set when other variables are entered later into the 
equation do not influence the'decision made from the initial entry of the 
first predictor set. For example, if the standardized partial regression 
coefficient of the first predictor set moves toward zero when later 
predictor sets are added to the equation, the situation is one of simple 
redundancy between the first and later predictor sets in accounting for the. 
criterion variance. If, on the^other hand, the regression coefficients of" 
the first predictor set changes sign. or increases, the relationship between 
the first and later predictor sets is one of suppression (see Cohen & Cohen 
(1975) for further discussion of this issue). With respect to the present 
TAEG study, examination of the standardized partial regression coefficients 

.^Forward stepwise regression selects from a group of predictors the one 
predictor at each stage that has the largest squared semipartial 
correlation coefficient, and hence makes the largesJt.xontribution to R2. 
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of the first predictor set oAmethods of instruction revealed no significant 
or relevant changes when the predictor set of ability level and training 
tasks was entered later in the\equation. Thus, entering methods of 
instruction into the equation f jrst (based on the tasking questions) did not 
, result in redundancy or suppressive effects not explained by main effects or 
■ interactions. \ ' ' 

' \ 
EXPLANATION OF REGRESSION TERMS COMMON TO ALL EQUATIONS IN THE MODEL 

r2 refers to the proportion o*\ the criterion variance shared with the 
optimally weighted predictor variables. 

\ R2 Increment is the cumulative increase in the proportion of criterion 
variance explained due to the addition of each set of predictors entering at 
the point in the equation, beyond the criterion variance already accounted 
for by the previously entered sets of predictors. These increments in R2 
are squared semipartial correlation coefficients. They represent the unique 
contribution of a predictor to R2 in the context of the remaining predictors 
(expressed as a proportion of the total criterion variance, since the 
effects of the remaining predictors have been removed fr,bm the specific 
predictor but not from the criterion). \ 

\ . 

B refers to the raw score partial regression coefficient. B is the 
constant weight by wh.ich each value. of a predictor is/to be multiplied in 
the equation and,reflects the average or expected, chang'e in the criterion 
value for each unit increase in each predictoV when the values of the 
remaining predictors are held constant. \ / 

\ ' " 

C is the regression constant or Y intercept for each equation, and 
serves to make appropriate adjustments for differences between the mean 
predictor values and mean criterion values (e.g., gives the predicted 
criterion value when predictor values=0; i.e. , '.is, the mean of the criterion 
variable). 

A 

Y refers to the predicted value of the criterion variable. In the 
regression model, Y represents predicted values for each of the criterion 
variables used in the present study. 

: . X refers, in general, to a set of predictor variables. 
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SPECIFICATION OF THE MODEL 



The number preceding each equation refers to the order (step) in which 
each set of predictors was entered into the regression equation. 

1. Y=Y"BiXi + C 

where X-j is composed of two coded predictors each 
representing different contrasts between methods 'of 
\ instruction. 

where Xj is composed of six continuous predictors, 
one representing AFQT and the remaining, five 
types" -Of training .tasks in percentages of task 
taught in school. ... 



A 

3. Y = 



4. 



5. 



where X|< is composed of 12 two-way interaction terms, 
10 of which were formed from two predictors of methods 
of instruction and five predictors of training tasks, . 
and two from two predictors of methods of instruction 
an'd one predictor of abtlity level. 



Y = y-BiXi ^jXj + VBk*k +VBlXl + 
L^U2 2jhU Lk=l,12Ll=l,5 



where X] is composed of five two-way terjns that were 
formed from one predictor of ability level and five 
predictors of training tasks. 



£-.1=1,2 £_j=l,6 Z^k=l,12 Z_ 1=1,5 Z_m=l,10 



where X m is composed of 10 three-way interaction terms 
formed from two predictors of methods of instruction, 
one predictor of ability level, and five predictors of 
training tasks. 



6a 



where S is one coded predictor representing sex of 
graduate. 
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6b. Y 



Li=i;6 2^=1,12^1=1,5 LnWloLn^O 



where L n is composed of three coded predictors 
representing school locations. 
A - 
6c. Y =V^B 1 x 1 VBjXj +V B kXk + V^m + B6 +■ C 

2-i=i,2 Ljhie Lk=i,i2Li=i'5 Lm=i;io 

where G is one continuous predictor representing 
end-of -course grades. G was used as a predictor 
only when Y = TAS rating. 

A 

r 6d: Y ^BiXT^+V^jXj +5TB k X k +VB1X1 +V , B m X m + BT + C 
1 Z.i=l,2 LffllU Zjk=l,122_l=l,5 Z-m=l?10 

where T is one continuous predictor representing 
training time. T was, used as a predictor only 
when Y = TAS rating: or training cost. 

Predictor sets and individual predictor variables are delineated below. 

Xi = Tl, T2 where Tl = II (SP + CMI) vs CI; contrast coded as II, (SP = 

CMI = = -1, CI = 1, and T2 = SP vs CMI, coded SP = -1, 
CMI = 1, CI = 0. 

Xj = A, II, 12, 13, 14, 15 where 

A = AFQT Percentile 

11 = Percentage of Fact Task 

12 = Percentage of Category Task • 

13 = Percentage of Procedure Task 

14 = Percentage of Rule Task 

15 = Percentage of Principle Task 

Xk = Till, Til 2 , T1I3, T1I4, T1I5, T 2 U, 
T 2l2, T2I3, T2I4, T 2 I 5 , ATi, AT 2 

Xl = All, AI 2 , AI 3 , AI 4 , AI 5 

Xm = AT1I1, AT1I2, AT1I3, AI1I4, ATil5,-. 
AT2I1, AT 2 I2, AT2I3, AT2I4, AT2I5 

S = Sex of graduate; dummy-variable coded as male = 1, female = 0 

L n = LI, L2, L3; effects coded as 

LI = 1 if San Diego, CA 
and -1 if Meridian, MS 

L2 = 1 if Great Lakes, IL * 
and -1 if Meridian, MS 
L3 = 1 if Millington, TN 
and -1 if Meridian, MS 

' 120 
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6 = Average end-of -course grades (percent correct) 

T = Training time (contact hours) , 

cot* Kn C i! men ^ l r Y R \ acc 2 u ? ted for by the Piously mentioned predictor 
sets can be described by the following equation: . * 

R2 Y.X lX2 X 3 ...k = R 2 YXi + R 2 Y(X 2 .XD^ R2 Y(X 3 .XiX 2 ) + + R 2 Y(k.XiX 2 X 3 ...k 

mof Jifn^f ren ? nt ! 3 R * are square J l Wltlple semipartial correlation 
coemcients. Each R' increment is an increase to the proportion of criterion 

a C c r ou a !!?e e dTo? u K^ for V r en predictor Set be * ond » ha? h * s IlSiS bS 

S3S2i^t£ y P I °* S* 1 .* 1 ' 8 - sets hi * hQr U P ™ the hierarchy). 
IZl Zf IIS Til 1 ° f *? e lnc , rement in criterion variance accounted for by 
that set can not be influenced by criterion variance associated with 
subsequent sets (i.e., those lower in the hierarchy). 
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APPENDIX 6 

MISSING DATA ANALYSIS 

This appendix presents the findings (with accompanying statistical 
tables and breakdown) of the missing data analysis. The purpose of this 
analysis was to determine if graduates with "data present differed from those 
with data absent. Headings used in this subsection" are listed by the 
criterion variables. s 

CRITERION: PRESENCE VS. ABSENCE OF TAS RATINGS 

Only main effects (and no interactions) were found in the regression 
analyses presented in table 6-1. \ 



TABLE G-l. SIGNIFICANT RESULTS FROM REGRESSION ANALYSE^ OF 
ABSENT TAS RATINGS 



-Step 


Variables 


B 


F 


DF * 


R2 

Increment 


Overall F DF_ 


1 


II vs CI 

SP vs CMI (T2) 


-.16 
.13 


234.46 
184.89 


1/4003 
1/4003 


.0811 


176.54 2/4003 


2 


AFQT (A) 
Fact (11) 
Principle (14) 


.002 
.02 
-.003 


18.25 
56.02 
40.88 


1/3998 
- 1/3998 
1/3998 


.0375 


34.03 4/3998 



The following findings are listed by significant predictor variables. 

Method of Instruction . A larger percentage of TAS ratings was 
absent from both SP and CMI graduates than from GP graduates. TAS ratings 
were absent more from CMI graduates than from SP graduates (see table 
G-.?). 

■TABLE G-2. ABSENT TAS RATINGS BY METHOD OF INSTRUCTION " 



ERJC 





Original 
Sample 
Size 






Absent 
TAS Ratings 






N 




N 




% 


SP 
CMI 

GP - 

TOTAL • 


1451 
823 
1732 
4006 




560 
536 
495 
1591 




39 
65 

'29 

40 
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2. Ability Level . TAS ratings were absent more from data of 
graduates of extremely lower and upper ends of mental categories than from 
the middle mental categories (see table G-3). 

TABLE G-3. ABSENT TAS RATINGS BY ABILITY LEVEL 



- 


f 
i 


Original 
bamp le 
Size 


ADsent 
TAS Ratings 




AFQT 
Percentile 
Range 


Mental Category 


N 


N 


% 


93-99 


1 


228 


113 


50 


' 65-92 


2 


1211 


^ 503 


42 


49-64 


3'U 


1608 


586 


36 


31-48 


3L 


828* 


341 


.41 


21-30 


4U '. ' 


36 


22 


61 


10-20 


' 4L 


1 


0 


O 




• Missing AFQT 


- 94 


26 


28 




Total 


4006 


1591 . 


40 



3. Training Task . More TAS ratings were absent from graduates who 
attended courses that taught a relatively smaller percentage of fact or 
rule Gaining tasks than those that taught a larger percentage of these tasks 
(see table G-4>. - . ^ < ' 

'? . JABLE G-4. ABSENT TAS RATINGS BY' TRAINING TASK 



Percent Task Taught in'«Cpurses 




0 




1-25 26-50 


51-75 


76-100 


-Task 


N % 


N 


% . N % 


N % 


N % 


Fact 


1036 34 


555 


56 - 






Rule 


733' 40 


670 


46 133 31 


23 12 


' 32 29 

1 
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CRITERION: PRESENCE VS. ABSENCE OF TINE TO COMPLETE (STUDENT CONTACT HOURS 
BASED ON SCHOOL RECORDS) 

Only main effects (and no interactions) were found in the regression 
analysis presented in table G-5. 



TABLE G-5'. SIGNIFICANT RESULTS FROM REGRESSION ANALYSES OF ABSENT 
TIME TO COMPLETE THE COURSE (CONTACT HOURS) DATA 



Step 


Variables 


B 


F 


DF 


R2' - 
Increment 


Overall F 


DF 


1 


II vs CI (Tl) 


-.02 


11.66 


1/4003 


.0239 


49.18 


2/4003 




SP vs CMI (T2) 


-.05 


72.16 


1/4003 








2 


Category (12) 


-.002 


8.41 


1/3998 


.0528 


228.64 


5/3998 




Procedure (13) 


-.002 


111.99 


1/3998 










Rule (14) 


-.004 


189.64 


1/3998 









The following findings are listed by significant predictor, variables. 

!• Method of Instruction . A larger percentage of time to complete 
data was absent from SP and CMI graduates than GP graduates. Time to 
complete data were absent more from SP graduates than from CMI qraduates 
(see table G-6). ; . 



TABLE G-6. ABSENT TIME TO COMPLETE DATA BY METHOD OF INSTRUCTION 





Original 
Sample 
Size 


Absent 
Time to Complete 






N 


N 


% 


SP 


1451 


168 


12 


CMI 


823 


20 ■ 


2 


GP 


1732 - 


f ,68 

k 


4 




Total 4006 


256 


6 
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2. Training Task , More time to complete data were absent from graduates 
who attended courses that taught a relatively smaller percentage of category, 
-procedure, or rule training tasks than those that taught a larger percentage of 
those tasks (see table G-7). — 



TABLE G-7. ABSENT TIME TO COMPLETE DATA BY TRAINING TASK 



Percent Task Taught in Courses 
0 1-25 . 26-50 51-75 76-100 



Task 


N 


% N 


% 


N 


% 


N 


% 


. N 


% 


Category 


26 


4 219 


7 


11 


14 










Procedure 




68 


9 


1 


% 1 


- 141 


15 


46 


2 


Rule 


128 


7 105 


7 1 


23 


5 











CRITERION: PRESENCE VS. ABSENCE OF END-OF-COURSE GRADES 

Only main effects (and no interactions) -werefound in the regression 
analyses presented in table 6-8. 

TABLE G-8. SIGNIFICANT RESULTS FROM REGRESSION ANALYSES OF 
ABSENT END-OF-COURSE GRADES. 



Step 


" Variables 


B 


F 


DF 


R2. 
Increment 


Overall F 


DF ' 


1 


II vs CI (Tl) 


.14 


196.28 


1/4003 


.1197 


272.38 


2/4003 




SP vs CMI (T2) 


-.21 


440.00 


1/4003 








2 


Category (12.) 


-.01 


.270.82 


1/3998 


.6453 


337.13 


5/3998 




Procedure (13) 


-.01 


1242.45 


1/3998 






i 




Rule (14) 


-.01 


692.24 


1/3998 
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The following findings are listed by significant predictor variables, 
graded was- f£spnt^^ S r£' iCt ^" \ A larger Pontage of end -of -course 




gradua es I Sue^ded c oupes 5t°r 9r ? deS Were abs * nt *™ 

procedure or rule Usks Z a «2Spi nf ht a ; re ^ vel y larger percentage of 
courses that taught smaller^ or llllZ E rce " ta 9| 2 ca1te 9 or y tasks than 
(see table G-10) * rger amounts of these ta sks, respectively 




Category 


97 


16 


1854 


56 


11 


14 




io ii X 


Procedure 






253 


32 


2 


1 


374 


39 1333 65 


Rule 


1136 


62 


387 


27 


254 


59 


185 


100 
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